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I. INTRODUCTION 


This thesis proposes the use of current Internet technology to develop a global 
collaboration environment to replace the traditional methods for conference planning and 
organization. An empirical prototype was developed and used to coordinate the 1998 IEEE 
International Symposium on Circuits and Systems (ISCAS 98) and to demonstrate and 


evaluate the effectiveness of this methodology. 


A. BACKGROUND 


Manual, labor intensive methods are traditionally used to plan conferences and 
symposiums. A large portion of the coordinating effort is attributable to the paper 
submission and review processes. The submission process begins with prospective authors 
mailing multiple copies of their proposed papers to the conference planning committee. 
The papers are received, verified, recorded and sorted by administrative personnel. An 
acknowledgment is mailed to each author to confirm their submissions. The review process 
begins by mailing multiple copies of the received papers to various designated reviewers 
along with review forms for their use in evaluating the papers. The reviewers read the 
papers before completing and returning the forms. The completed forms are collected and 
used by the Technical Program Committee (TPC) to select the papers which will -be 
presented during the conference. All of the accepted papers are then scheduled into sessions 
by topic. The authors of the accepted papers are notified to submit final versions of their 
papers and to register for the conference. These final papers are then sent to be published in 
the conference proceedings. 

In recent years, electronic means have been incorporated into various individual 
aspects of the overall conference planning process. File Transfer Protocol (FTP) servers 
have been used to support the receipt and distribution of electronic papers while formatted 
e-mail review forms have been used to support the review process [Ref. 1]. Also, electronic 


mail (e-mail) has been used for receiving paper submissions as well as for distributing the 


papers to the reviewers [Ref. 2]. Although these electronic techniques have enhanced some 
of the conference processes, there has been no effort to provide an integrated system using 
interactive applications for automating information storage and retrieval in most or all 


conference processes. 


B. AREA OF RESEARCH 


The primary objective for this thesis is to evaluate the feasibility of using the World 
Wide Web (WWW, or Web) to deploy an integrated suite of conference organization and 
collaboration applications. The scope of this research will include the use of a database as a 
back end, or data source, for the Web applications to provide automated on-line data storage 
and retrieval. This will result in the integration of a Web server with a database server to 
support the interactive application requirements which are essential for providing an on-line 


conference collaboration environment. 


C. PRIMARY RESEARCH QUESTIONS 


@ Which conference processes can be Web-enabled? 

@ Which conference processes are best suited to Web deployment? 
@ What are the hardware and software support requirements? 

e Is scalability an issue with regard to hardware and software? 


e Are there any special considerations for a Web deployed system? 


D. DISCUSSION 


Using current Internet technology, an interactive and automated system will be 
developed and evaluated for effectiveness in replacing the traditional methods used for 
conference organization - particularly those related to receiving, processing, distributing and 


reviewing papers submitted for conference presentation. The 1998 IEEE International 


Symposium on Circuits and Systems (ISCAS '98) will be hosted by the Naval Postgraduate 
School in May and June of 1998. The Circuits and Systems Society is the oldest and largest 
professional organization in the IEEE. Their annual conference is one of the largest of its 
kind. Several of the ISCAS '98 conference support processes will be integrated into a Web- 
based prototype which will be used to coordinate the ISCAS conference planning activities 
over the Internet. 

The objective of the ISCAS ‘98 prototype is to pipeline all of the conference 
organization processes into an integrated on-line collaboration environment. This prototype 
is designed to use a Web interface to provide a network distributed paper submission and 
review system for managing the various aspects of conference organization. The prototype 
allows any user (author, reviewer or committee member) to perform required tasks and/or to 
access essential information from globally remote locations via the Internet with the use of 
standard Web browser software. This thesis provides an overview of the ISCAS prototype 


development and analyzes the results and conclusions derived from the research efforts. 


E. BENEFITS OF STUDY 


This research will provide lessons learned and an initial prototype from which a 
globally networked conference collaboration spsiamn can be developed. Many of the 
processes related to the ISCAS prototyping effort can also be abstracted into other 
Internet/Intranet applications. In particular, the fundamental ISCAS applications are 
directly related to Electronic Commerce / Electronic Data Interchange (EC/EDI) functions 
and can easily be applied to highly formalized government activities like the DOD's sealed 
bidding process. The prototype’s integration of database and Web technology will further 
demonstrate the capability for converting paper-based forms activities into interactive Web- 
form based processes which lead to a disintermediation of administrative requirements. 
For the Naval Postgraduate School, two such activities which could benefit from this 


research are student course registrations and submission of Student Opinion Forms (SOFs). 


F. SUMMARY OF REMAINING CHAPTERS 


Chapter II describes the resources and identifies the requirements for the ISCAS 
prototyping effort. The chapter includes a discussion on previous work for the conference 
in addition to the user and process requirements for interacting with the prototype. Chapter 
III provides some background on system implementation and migration issues. It also 
contains a detailed description of the core applications which were developed for the ISCAS 
prototype. An analysis of the ISCAS prototype is provided in Chapter IV. The analysis 
includes an overview of the results, provides a discussion on the various lessons learned, 
and details recommendations for enhancing the prototype. Finally, conclusions for the 
thesis research questions listed above are addressed in Chapter V. Several 


recommendations for further research are also discussed. 


Il. RESOURCES AND REQUIREMENTS 


This chapter describes the work which had already been completed by the ISCAS 
planning committee before the prototyping effort began. Included is a discussion on the 
resources required to build and support the Web server for the ISCAS prototype. 
Additionally, the user and process requirements deemed essential to the prototyping effort 


are described in detail. 


A. BACKGROUND 


The ISCAS °98 conference management process was well underway prior to 
beginning any requirements analysis for developing the on-line collaboration system. The 
pre-existing work would serve to provide the preliminary requirements used during the 
initial stages of prototype development. This section provides a background discussion 
covering the requirements derived from previous work and pre-existing resources. 


1. First Call For Papers 


A single page flyer distributed by the conference planning committee would become 
the most important resource used to determine prototype requirements. The ISCAS °98 
conference had been originally announced through a distribution of small color posters. In 
addition to the conference location and schedule, the posters provided a point-of-contact 
and a temporary World Wide Web (WWW or Web) address. These posters were later 
followed by a more detailed flyer titled: “1998 IEEE International Symposium on Circuits 
And Systems: First Call For Papers.” This flyer would form the primary content for the 
ISCAS Web site and would lead to the preliminary requirements specification used in the 
development of the prototype’s initial on-line paper submission application. 

The “First Call For Papers” invited prospective authors to submit extended 
summaries of papers reporting their original work in all areas of circuits and systems 
research. Paper summaries for regular sessions were required to be submitted to the 


Technical Program Committee (TPC) for review. There were 54 proposed topics listed in 


the flyer; however, papers were not limited to those topics. Figure lis a list of the original 


topics provided in the “First Call For Papers.’ 
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Additional proposals for special sessions, 


plenary sessions and short courses were required to be submitted for review directly to the 


1 Analog Circuits and Discrete Systems 6 Digital Signal Processing 


1.1 
le2 
13 
1.4 
1.5 
1.6 


Analog circuits and systems 

Switched capacitor/current techniques 
Amplifiers, oscillators 

Solid state circuits 

Analog and mixed signal processing 


Data conversion and S-D modulation 


2 Circuit Theory and Power Systems 


2a 
Zen 
ZS 
2.4 
2 
3 VLSI 
er 
a2 
a5 
3.4 
3 


Linear and nonlinear circuits and systems 
Chaos and applications 

Distributed Systems 

Power distribution systems 


Power electronics and systems 


Analog 1Cs, digital ICs, and ASICs 
Low voltage/low power ICs 

GaAs and high speed 1Cs 

Fault tolerant systems 


Testing: analog, digital, and mixed 


4 Computer Aided Design 


4.1 
4.2 
4.3 
4.4 
4.5 
4.6 


Circuit layout 

Modeling and simulation 

Optimization techniques 

Large scale networks 

Circuit modeling for electronic packaging 
New CAD tools 


5 Neural Systems 


SEL 
a2 
5.3 
5.4 
5-5 


Neural networks 

Cellular neural networks 
Biological computing 
Fuzzy circuits and systems 


Biometrics 


6.1 
6.2 
6.3 
6.4 
6.5 
6.6 


Digital filters and filter bank design 
Wavelets, multirate signal processing 
Parameter estimation 

Adaptive signal processing 
Multidimensional systems 


Fast algorithms 


Multimedia and Video Technology 


4) 
TZ 
ips 
74 
We 


Speech processing and coding 

Image processing and coding 

Video coding & motion compensation 
High definition television 


Multimedia systems 


Communication Circuits and Systems 


8.1 
8.2 
8.3 
8.4 
8.5 


Signal processing for communications 
High frequency circuits 
Wireless/mobile communications 
Optical communications 


Underwater communications 


Computer Communication Systems 


9.1 
2 
2, 
9.4 
1S) 
9.6 


Applications 


10.1 
10.2 
10.3 
10.4 
10.5 





‘High speed modems 


Broadband ISDN 
Asynchronous transfer mode 
Speech and video modeling 


Teleconferencing & distance learning 


SONET and optics circuits 


Sensors 

Robotics 
Electromechanical systems 
Dual use technology 


Space systems design 


Figure 1. Prospective Regular Session Topics 


respective committee chairs. Accepted proposals for all sessions would ultimately require 
final paper submissions; however, since the regular sessions would comprise the majority 
of the conference, the initial Web applications would be designed exclusively for the 
proposed regular session papers. A separate ‘final’ paper submission application would 
later provide for the submission of all final papers. 

The requirements for submitting extended paper summaries were also included in 
the “First Call For Papers.” These included submission and notification deadlines, required 
paper summary length, and required cover sheet information. Figure 2 contains a 
reproduction of the paper submission requirements. There were two methods for 
submitting the papers. First, a postal address was provided to allow for mailed submissions. 


Postal submissions would require five hard-copies of the paper summary in addition to an 


Authors are invited to submit extended summaries (1000-2000 words) of their papers along with a cover sheet 
for review by the Technical Program Committee. Submissions can be sent to: 
ISCAS98 Technical Program Co-Chairs 
c/o Department of Electrical and Computer sical 
Naval Postgraduate School 
Monterey, CA 93943, USA 
Submissions are accepted in ONE of the following two formats only (No FAX submissions please): 
*PDF/ Postscript submissions via WWW« at http:/ /iscas.nps.navy.mil/submit/ 
*Five copies of hard-copy submissions by postal mail; please include PDF/PS/DVI file on floppy disk 


Submissions should include: 
1) A Cover Sheet containing: 
(i) Title of proposed paper, authors’ names and affiliations; 
(ii) Postal address, phone and FAX numbers, and e-mail address of the contact author; 
(iii) Paper category number (from the topic list) that best describes the paper; and 
(iv) Choice of presentation (lecture or poster) 
2) 1000-2000 word extended summary that includes: 
Paper title, authors’ names, affiliations, and addresses; and 
Paper category number (from the topic list) 
Once accepted, authors will be asked to prepare a four-page camera-ready paper for the symposium 
proceedings. 
AUTHORS’ SCHEDULE 
Deadline for Submission of Summaries October 1, 1997 
Notification of Acceptance December 15, 1997 
Deadline for Submission of Camera-ready paper January 30, 1998 





Figure 2. Extended Summary Submission Requirements 


electronic version on floppy disk. Second, a new and permanent WWW address provided 
access to the formal ISCAS Web site which also contained a file upload application that 
would allow for electronic paper submissions over the Internet. Electronic submissions 
would be limited to either Adobe’s Portable Document Format (PDF) or Postscript file 
format as will be discussed further below. 


pap Electronic Submission File Formats 


The ISCAS planning committee had previously determined that use of the Portable 
Document Format (PDF) from Adobe Systems Incorporated would be an essential 
requirement for the ISCAS prototype. 

Adobe’s PDF lets authors publish electronic documents that preserve the 

look and feel of documents they create. [Ref. 3] 

This meant that electronic versions of final papers in PDF format would look exactly like 
the original printed versions. In fact, the printed versions for the conference proceedings 
could be printed directly from PDF files and would still look as if they had been printed 
from within the original authoring application. This capability would allow for the 
electronic submission of papers and would eliminate the need for authors to mail paper 
copies to the ISCAS TPC. 

A PDF file viewer application would be required in order to open, view and print 
the received papers. Adobe provides a free viewer application called Adobe Acrobat 
Reader. This application is available for free download from many different locations on 
the Web, including Adobe’s own Web site at www.adobe.com. The viewer application is 
also typically included with the distribution of many software titles, especially the major 
Web browser applications. The Acrobat Reader application would also be required for and 
included on the CD version of the conference proceedings. 

The Web browser plug-in included with the Acrobat Reader application would be 
another key piece of software for the prototype. This plug-in allows for the viewing and 
printing of PDF files over the Internet from directly within a Web browser application. 


This capability would allow the ISCAS prototype to easily distribute the PDF papers for on- 


line review during the review and selection process. Although all reviewers would need the 
Acrobat Reader application and the browser plug-in to be installed on their computers, use 
of the software would eliminate the need to copy and mail out papers to each of the over 
200 conference paper reviewers. 


3. The Initial ISCAS Web Site 


A Web site had already been established for the ISCAS conference prior to the 
commencement of prototype planning and development. The initial ISCAS Web site was 
installed on an Intel 486 Personal Computer (PC) running the Linux operating system and 
Apache Web server software. A mail server application ran concurrently with the Web 
server. The mail server facilitated electronic mail (e-mail) among committee members and 
provided broadcast capability for mass announcement electronic mailings. The Web site 
was essentially an electronic version of the “First Call For Papers.” It displayed a basic 
home page with links to a few additional pages. The site advertised the 1998 ISCAS 
conference and provided links to electronic versions of the information contained in the 
‘First Call For Papers” flyer. The Web site also contained an on-line form which supported 
the electronic submission of regular session paper summaries; but it did not provide a 
mechanism for automatically accumulating and organizing paper and author information 
into a database. 

A Common Gateway Interface (CGI) application was being used to enhance the 
functionality of the ISCAS Web server. 


A CGI program provides a powerful capability which greatly expands the 
role of a Web server to allow for graphical fill-out forms whose information 
can be sent back to the CGI for further interpretation. [Ref. 4] 


The CGI being used was a Perl program language script that had been created to process 
the existing Web form in order to allow for the electronic submission of papers. Whenever 
an author used the Web form to submit a paper along with the required contact information, 
the Web server would pass the incoming data to the CGI for processing. The CGI would 


then handle the tasks of uploading the paper to the server and e-mailing the information 


from the Web form to an administrator. The e-mail message contained contact author and 
paper information which had to be manually entered into an existing Microsoft Access 
database by administrative personnel. Any additional information required for the database 
then had to be pulled from the cover sheet of the uploaded paper and entered into the 
associated paper record within the Access database. The schema for this pre-existing 
Access database would provide the preliminary data model which would be used to develop 
a relational database schema for the advanced CGI applications to be built for the prototype. 
Prior to completing a fully automated paper submission CGI for the ISCAS prototype, 60 


paper summaries were submitted using the initial Web form and CGI configuration. 


B. THE ISCAS WEB SERVER 


A variety of hardware and software resources would be integrated to develop and 
support the prototype server. This section discusses the pre-existing and procured server 
resources which would be essential to the ISCAS prototype. A discussion regarding 
implementation and migration is saved for Chapter III. 


Ls Web Server Resources 


The Web server for the ISCAS prototype would require a more robust and higher 
capacity hardware and software configuration than the Intel 486 PC could provide. The 
ISCAS planning committee had previously determined that a Microsoft Windows NT 
Server with a large hard disk storage capacity would be used to support the conference. A 
Dell PowerEdge 2100/200 MHz Pentium Pro server platform running Microsoft Windows 
NT 4.0 Server operating system would be acquired to provide substantial back end support 
to the Intel 486 PC. Netscape Communications’ Enterprise Server software would then be 
acquired to provide the necessary Web services on the NT server. The 486 PC would still 
be needed to provide mail and domain name services. 

The addition of Everyware Development Corporation’s Tango for Access, a 
database middle-ware CGI, would provide for direct interaction between the Enterprise 


Web server and the Access database, thereby allowing for automatic insertion and retrieval 


10 


of paper and author information using a standard Web browser. The Tango software 
consists of two applications, Tango Editor and Tango Server. Tango Editor is used to 
develop Web applications capable of interacting with an Access database. Tango Server, 
the actual CGI, extends the capability of a Web server to process the Web applications 
developed using the editor. Use of the Tango CGI would eliminate the need to have the 
Perl script CGI e-mail the received Web form data to administrative personnel for manual 
entry into the Access database; however, the Perl CGI would still be required to perform file 
uploads. Thus, the existing Perl script would require modification to remove the e-mail 
feature after porting it over from the 486 PC to the new NT server. Its only remaining 
functionality would be to upload the paper summaries from the client platforms to the NT 
server. 


a Support Resources 


Adobe’s Acrobat Exchange software would be required to support the receipt of 
papers in Postscript file format. Acrobat Exchange software provides the ability to create, 
view, edit and print PDF files. Its companion application, Acrobat Distiller, is capable of 
converting batches of Postscript files into PDF files. The Acrobat Exchange software 
would normally be required for any author who wanted to submit a paper in PDF format. 
Rather than forcing all authors to acquire Acrobat Exchange, the TPC would accept 
submissions of electronic papers in the Postscript file format; however, this would require 
ISCAS administrative personnel to convert the received Postscript files into PDF files. 
Further, most standard methods of encoding and compression would be allowed in order to 
minimize file transmission times. Papers received in any of these various formats would 
first have to be decompressed and/or decoded before being converted to PDF. Once a paper 
had been received and converted to PDF, it could then be placed into the Web server’s on- 
line paper directory and made available to the reviewers via the Internet. 

Several additional resources would be needed to provide background support for the 
prototype. An uninterruptible power supply (UPS) would be needed to keep the Web server 


on-line in order to maintain the reliability and integrity of the system. A tape backup 


I] 


system based on Seagate’s Backup Exec would be used to safeguard the contents of the 
Web site, the database, and the uploaded papers in the event of a catastrophic disk failure. 
Two scanners would be needed to scan in any mailed paper summaries which were not 
accompanied with an electronic version on disk. A Postscript laser printer would be 
required to print out any problematic files which could not be electronically converted into 
PDF format. These printouts would then be scanned into electronic PDF format using the 


Adobe Acrobat Exchange software. 


C. PROTOTYPE REQUIREMENTS 


This section discusses the user and process requirements which were identified 
primarily from the “First Call For Papers” and through interaction and discussion with the 
ISCAS committee members. Since the prototype would actually be used in support of a 
real-world conference, a build-and-fix prototyping methodology would be used. This 
approach would result in additional requirements being identified through user feedback 
gained from interacting with the prototype. 


1. User Requirements 


User requirements would determine which applications would be required and when 
they would be needed. Throughout the course of the prototyping effort, five categories of 
clients, or user groups, would be identified. Each user group would require different access 


levels and specific types of interaction with the prototype. 


a. Paper Authors 


Authors represented the largest user group (well over 2000). They would 
require general public access to the prototype in order to submit their papers on-line. They 
would also require the capability to resubmit their papers in the event of a transmission or 
other problem during the submission of their original papers. Final paper submissions, 
including abstracts, would also be required for those authors whose original submissions 


had been selected for presentation at the conference. 
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b. Paper Reviewers 


Reviewers (over 200) would need to conduct on-line reviews of the papers. 
This would require access to the PDF papers and to restricted portions of the on-line 
database for the related paper records. The reviewers would be required to submit on-line 
review forms for each paper they reviewed. A protected access mechanism would be 
required to prevent general public access to the on-line PDF papers and to the database. 


This would also ensure that only designated reviewers could submit on-line reviews. 
C. ISCAS Planning Committee 


Committee members (approximately 20) would require on-line access to 
retrieve paper and author information from the database in addition to submission statistics. 
This information would be needed to respond to queries from authors and to coordinate the 
selection and assignment of paper reviewers. In addition, the Technical Program 
Committee (TPC) members would require access to all of the on-line reviews submitted by 
the paper reviewers. The reviews would be used during an on-line selection process to 
determine which papers would be accepted for presentation at the conference. After the 
selection process, the TPC members would require access to an on-line scheduling 
application in order to assign each of the accepted papers for presentation during one (and 
only one) of the 180 sessions planned for the three day conference. The sessions also had to 
be organized and scheduled for the conference; however, this process would prove difficult 
to constrain in a manner which would allow efficient on-line coordination. Due to time 
constraints and process complexity, session scheduling would be handled manually during a 


two day meeting of the TPC. 


d, Administrators 


Five Administrative personnel would manage all related aspects of 
receiving, confirming and acknowledging the papers. This would include manual entry of 
papers received via postal mail, converting all received files into PDF format, validation of 


the database and all received papers, acknowledging the condition of received papers to the 
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contact authors, and notification of authors for the accepted papers. Administrators would 
also require complete access to the database, to include a capability to delete invalid and 


duplicate records. 


é. Developer 


The prototype developer would design, build, test and maintain the 
applications and the underlying database in support of all users. Access would be required 
to all applications and to the entire database. 


2 Process Requirements 


Many processes would be identified in support of the user requirements described 
above. Each of the core processes described below would result in a specific application 
that was developed for the ISCAS prototype and made available for use over the Internet. 


A detailed discussion of those core applications will be left for Chapter III. 


a. Paper Submission 


To support an on-line paper submission process, an application would have 
to meet a variety of requirements. In addition to accepting extended paper summaries in 
electronic file format via the Internet, 1t would have to provide the capability for authors to 
submit their paper cover sheet data, author data and contact information. The submitted 
electronic file would have to be renamed using an auto-generated paper tracking number 
and stored in a protected receiving directory. The cover sheet, author, and contact 
information would need to be automatically stored in an on-line database using the assigned 
paper tracking number as an index. Further, a resubmit option would be required to address 
electronic transmission problems. Administrative personnel would also need to use this: 
application to load papers received by postal mail directly into the on-line system. This 
prototype application would have to be made available for general public access until the 


submission deadline had passed. 
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b. Logging In 


A login application would be necessary in order to provide access to the 
higher level functions required by reviewers, committee members, administrative personnel 
and the developer. When used to gain access to the system, this application would have to 
provide a welcome screen that displayed all functions for which a given user had access. 
This meant that users (other than authors) and applications would have to be assigned 
individual access levels to support a run-time comparison by this login application. A 
tutorial provided with the Tango for Access software would be used as the basis for 
developing the login application. The Tango Tutorial would also be used to guide the 
creation of user accounts as well as the assignment of user and application access 


levels.[Ref. 5] 


res Acknowledging Receipt of Papers 


An application would be needed to provide administrative personnel the 
capability to confirm that submitted papers were in good condition and were readable over 
the Internet by providing on-line links to the papers. The application would also have to 
support the automatic generation of two separate form letter style e-mail messages. The 
primary e-mail message would be for notifying contact authors that their submitted papers 
were received in good condition (acknowledgments). The alternate e-mail message would 
be for notifying contact authors to inform them of problems with their submissions and to 
request resubmissions (negative acknowledgments). Both e-mail messages would need to 
be auto-generated from an appropriate template using the information from the Access 
database. Each would require the inclusion of all identifying paper and author information. 
Additionally, the on-line form e-mail application would have to provide a custom editing 
feature to allow administrative personnel the capability to include specific comments in 
either of the messages, especially with regard to unreadable submissions. This application 
would also need to update the Access database to mark a paper record as having been 
acknowledged (Acked) or negatively acknowledged (NAcked) after transmitting the 


appropriate e-mail message. 
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d. Paper Review 


To support the review of papers over the Intermet, designated reviewers 
would require access to an application which could provide links to assigned papers for on- 
line and/or off-line reading. The application would also have to provide a capability for 
submitting an on-line electronic review form. The review form would need to accept 
criteria rankings, comments, and an acceptance recommendation (Accept, Not Sure, 
Reject). The data from the review forms would have to be automatically stored in the 


database for later use by the TPC during the paper selection process. 


e. Paper Selection 


The paper selection process would involve comparing and contrasting all 
reviews submitted for each paper in order to make an accept or reject decision. The TPC 
members would need an on-line application to support this process by allowing selection of 
papers for presentation at the conference. The application would require the capability to 
retrieve, list and sort reviews by paper number. Additionally, it would have to provide a 
form for making and/or changing the final acceptance decision for each paper. The 


database would also have to be automatically updated to reflect all final status decisions. 


f Presentation Scheduling 


Each of the accepted papers would have to be scheduled for presentation 
during one of the 180 sessions planned for the three day conference. This would require the 
TPC to first group all accepted papers by related subtopics to identify the themes for each of 
the sessions. The complexity involved in this process would require a two day TPC 
meeting to manually organize the sessions using a large wall as a layout to ensure that 
sessions with similar themes did not overlap. Once this was completed, an on-line 
application would be required to allow TPC members to schedule each individual paper into 
a session with an appropriate theme. The application would have to ensure that each 
accepted paper could only be scheduled once. It would also have to support removing a 


paper from one session so that it could be added to another. Further, the application would 
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need to provide a mechanism for labeling a session with a title, a session chair and a session 


co-chair. 


gZ. Final Paper Submission 


After the selection and scheduling of papers for presentation at the 
conference, POC authors of accepted papers would require notification to submit a final, 
camera-ready version of their papers for publishing. Address, paper and author information 
from the Access database would be used by administrative personnel to prepare formal 
letters using a word processor. The formal letters would include scheduling information 
and would direct the authors to submit their final papers on-line. The POC authors would 
also be notified, via a broadcast e-mailing, to check the ISCAS Web site for a list (by paper 
tracking number only) of all accepted papers and for a link to a final paper submission 
application. The address list for this e-mail message would be generated from the database. 
Most importantly, an on-line submission process, nearly identical to the extended summary 
submission process described above, would be required to support the submission of final 
papers. Some changes would be necessary, including the additional requirement to have 
authors include an abstract for their papers. The application designed to meet the final 
paper submission requirements would take into account any lessons learned from the 
Original submission application. It would also need to be linked to a new Access database 


from which the entered information could ultimately be published. 


h. Miscellaneous Support 


In addition to the core process requirements above, several support 
applications would be required to address needs specific to managing the prototype. Most 
of these applications would be specialized queries and single purpose utilities designed in 
response to individual requests from committee members and administrative personnel. 
Further, some special purpose applications were only required as a result of the migration 
from the original submission application located on the initial Web server. These 


applications would not have been required had the prototype been on-line from the very 
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beginning of the paper submission process. The following is a list of the more prominent 


support application requirements: 


An application would be required to provide a means for transferring the 60 
papers received using the original Perl Script CGI from the 486 PC over to the 
newer NT server. This application would allow administrative personnel to 
transfer the papers and enter the appropriate information into the database using 
the previously assigned paper tracking numbers. The first 60 tracking numbers 
for the new submission application would have to be reserved for this purpose. 


Since approximately fifty reviewers submitted papers for the conference, a 
special utility would be required to allow the TPC chair to tag their paper 
records and identify them as belonging to paper reviewers. This would prevent 
paper reviewers from being able to access their own records to recommend final 
acceptance. 


Several browser applications would be needed to support different database 
query capabilities. Administrative personnel and committee members would 
require paper access by paper number, by paper title and by author to respond to 
author trouble calls. These browser applications would also be used to locate 
duplicate records and to check for existing records prior to entering submissions 
received by postal mail. 


Also, various queries would be requested by committee members to generate 
one-time listings of records meeting certain criteria. Examples of particular 
requests include requirements for: a list of POC author e-mail addresses, a list of 
papers pending review, a list of papers pending scheduling, a preliminary 
conference schedule, and a list of accepted papers. On-line applications to 
support these types of queries could generally be completed in a matter of 
minutes. 


i. Other 


One of the early goals for the ISCAS prototype was to provide an on-line 


registration capability which would be integrated with the database to automate the 


registration process. This would have greatly simplified the coordination required with the 


publishing efforts to ensure that authors of published papers had in fact registered and paid 


for the conference. Due to time constraints and security concerns, an application for this 


process would not be included in the prototype’s application suite; however, the on-line 
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registration capability would be provided via an outsourcing agreement with a commercial 


provider. 


Is 





Il. THE ISCAS PROTOTYPE 


The ISCAS prototype is currently being used at the Naval Postgraduate School as a 
working prototype for coordinating and managing the IEEE’s 1998 International 
Symposium on Circuits and Systems (ISCAS ‘98). As stated previously, the objective of 
the prototype was to pipeline, into a single on-line system, all of the traditional conference 
organization processes. As a result, a variety of conference functions were converted into 
browser-based applications and made available over the Internet in order to provide a 
world-wide collaboration environment for the widely dispersed members of the committee 
and all other dbiiforciice panicisanel Of particular concern, were those functions related to 
receiving, processing, acknowledging, distributing, reviewing, accepting and scheduling 
papers that were submitted for conference presentation. Final paper submission, conference 
registration and publishing functions were also targeted for prototype inclusion. 

This chapter begins by describing the migration from the pre-existing server to the 
new server which provided the foundation for the ISCAS prototype. Each of the core 


applications developed during the prototyping effort are then described in detail. 


A. SERVER MIGRATION 


Prior to deploying an initial production version of the ISCAS prototype, Web 
services had to be migrated from the 486 PC to the new server. This started with the 
development of a new Web site for the NT server. Information was pulled from the “First 
Call For Papers” flyer and from the initial Web site located on the 486 PC. Page design, 
HTML coding and site publishing were completed using GoLive CyberStudio Web design © 
software on an Apple Macintosh PowerBook 5300 running Apple’s MacOS 8.0 operating 
system. The published site was then loaded onto the NT server. To maintain mail services 
on the 486 PC and to avoid changing the well-advertised domain name 
(www.iscas.nps.navy.mil) for the original Web site, the new site’s home page was also 


loaded onto the 486 PC in place of the pre-existing Web site. The new home page 
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contained an HTML frame set with links to the remainder of the new Web site and its full 
structure, all of which resided solely on the new NT server. At this point, HTML pages 
from the Web site were being served from the NT server while the 486 PC continued to 
handle the paper uploads. This setup allowed for a distribution of server resources between 
the old and new servers while providing for a smooth testing and migration path to the 
eventual prototype on the NT server. The only visible difference to clients browsing the 
Web site was the new design and layout of the Web pages. All potential confusion with 
Web addressing and new Web site announcements had been avoided. 

Before development could begin on the Web applications, a database back-end for 
the ISCAS conference was prepared and loaded onto the NT server. The original Microsoft 
Access database was used along with the “First Call For Papers” flyer to identify which data 
would be required to support the Web applications. A new relational database schema was 
then developed to provide greater flexibility in application development and to minimize 
the limitations and data redundancy problems typically associated with the mostly flat-file 
format of the existing Access database [Ref. 6]. The Tango Editor application was then 
used to link to the newly created Access database in order to begin development on the first 
Web application. 

The new extended summary submission application (discussed below) was the first 
Web application completed and linked to the new ISCAS Web site. It represented the final 
step in migrating to the NT server. The modified NT Perl script was used in conjunction 
with this new submission application (via the Tango Server CGI) to upload the papers to the 
NT server. Communications between the Perl script CGI and the Tango Server CGI were 
conducted through a combination of HTTP redirect statements and environment variable 
exchanges. This allowed the two CGlIs to pass data for uploading the papers and to 
coordinate database updates after a successful upload without requiring additional 
interaction by the clients. With the exception of electronic mail services, all server activity 
was then being handled by the NT server. The final modification required on the 486 PC 


was to set a server redirect for any clients to be automatically and unknowingly transferred 
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to the NT server when using the ISCAS Web address. No further system configuration 
changes were required. From this point on, additional applications were developed, tested 
and linked to the Web site on a pmiority basis in order to meet the necessary user 


requirements for interacting with the ISCAS prototype. 


B. PROTOTYPE PHASES 


The various conference functions could be separated into four distinct phases for the 
overall prototyping effort. The first phase was called the Initial Submission Phase. This 
phase included the creation and distribution of the First Call For Papers, the submission of 
extended summaries for committee review, and the acknowledgment of received 
submissions. Next was the Selection Phase, which included the distribution of extended 
summaries to reviewers, the collection of review forms, the selection of conference papers, 
and the scheduling of selected papers for conference presentation. Then came the Final 
Submission Phase, which included the notification of accepted and rejected authors, the 
submission of final (camera-ready) papers, and the acknowledgment of received papers. 
The fourth and final phase would be called the Publishing Phase and would include 
conference registration and publishing of the conference proceedings. 


1. The Initial Submission Phase 


a. Background 


In a traditional conference, a First Call For Papers would be distributed to 
signal the start of the Initial Submission Phase. Prospective authors would then submit their 
proposed papers for review by mailing them to the conference organizing committee at the 
host institution. Five or more copies of each paper would be received, verified, recorded, 
sorted, and filed for distribution to reviewers by administrative personnel. This would 
include manual data entry into a conference database and assignment of individual paper 
tracking numbers. After receiving the prospective papers, a formal acknowledgment would 


be mailed to all POC (point-of-contact) authors to confirm the receipt and condition of each 
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paper. In recent years, limited forms of electronic submission, incliiding the use of e-mail 
and FTP (File Transfer Protocol), were allowed; but the administrative personnel still had to 
perform all of the associated functions for this phase. The SPAA ’98 conference is a good 
example of a conference using e-mail to support the electronic submission of papers. Their 
Web site is located at “http://sigact.csci.unt.edu/~spaa98/SPAA98 .html#email’’. [Ref. 2] 
The ISCAS prototyping effort proved to be extremely successful in fulfilling 
the functional requirements of the various processes involved in the initial submission 
phase. Development of the prototype allowed for full automation of all functions during 
this phase except for the verification and acknowledgment of the received papers. No 
copying, sorting, filing, or data entry would be required for the papers submitted via the 
web. The entire phase was designed to be completely paper-less. Administrative personnel 
would only be responsible for decompressing and converting non-PDF files into PDF 
format, moving all received papers (PDF files) from the receiving directory into a 
designated on-line directory, and acknowledging the papers via e-mail by using an on-line 
form letter. Everything could be done remotely, via the Internet, using standard web- 
browser software except for the file conversions and the file transfers between directories. 
These activities required physical access to the server. (Note: Advanced scripting 
capabilities are available which could also fully automate the file decompression, file 


conversion and file transfer requirements; however, none were incorporated into this initial 


prototype.) 
b. The Paper Submission Application 


The ISCAS 798 First Call For Papers (discussed in the previous chapter) was 
distributed to prospective authors and appropriate institutions through the use of mailing 
lists which are routinely maintained by IEEE. Prospective authors were directed to the 
ISCAS °98 home page on the web for the on-line electronic submission capability. Figure 3 
displays a recent copy of the home page. During the Initial Submission Phase, a “Submit” 
link was provided in the left hand menu under “Papers”. Clicking on the link with a mouse 


would bring up a new page (Figure 4) with some general submission guidance and a button 
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titled “Submit a Paper’. Pressing the button would start the “Paper Submission 
Application.” 

The form displayed in Figure 5 would be displayed in an author’s browser to 
start the paper submission process. This form collects the POC Author and paper 
information, which will be automatically stored in the Microsoft Access database by the 
Tango CGI when the “Submit Paper Information” button is pressed. After storing the 
information, the CGI sends the form displayed in Figure 6 back to the client’s browser. 
This form appears with the “Last Name” and “Email Address” fields already filled in with 
the POC author information. It collects additional personal information for the author 
which will be stored in the database after pressing the “Submit Author Information” button. 
If “Is there another author?” has “Yes” selected and the bottom two fields (Last Name and 
Email Address) are filled in, then the form will reappear to repeat the process for each 
additional author. If “Is there another author?” has “No” selected, then the “Paper 
Submission Form”, shown in Figure 7, will be displayed to allow electronic submission 
(upload) of the paper. 

The “Paper Submission Form” actually serves as the interface to the paper 
upload CGI. This form contains a “Tracking Number’ which has been automatically 
assigned to the paper by the Tango CGI. The author will need this number for any future 
correspondence and for resubmission of the paper if necessary. After pressing ‘the 
“Browse...” button to select the appropriate file, the author can start the upload process by 
clicking on the “Press Here” button. The NT Perl CGI then receives the incoming file, 


renames it using its assigned tracking number, and stores it in a designated 
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Figure 3. Home Page for ISCAS ’98 Web Site 
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ISCAS '98 Paper Submission 





® Please be sure your proposed paper's cover sheet contains: title, authors' names, and 
affiliations plus postal address, telephone, FAX and e-mail of the contact author. 

® Your file should only be in PDF or Postscript format. 

® Postscript files larger than 350KB should be compressed using gzip, UNIX compress, or pkzip. 
Please include a note indicating the type of compression you used in this case. 

®@ Please be patient. Depending on your connection, it can take up to 15 minutes to complete the 
actual file transfer. 

® If you successfully upload your paper, you will see anew page with a serial number. Please 
use this serial number when referring to your paper in future correspondence. 


® The organizing committee can not be responsible for submissions that do not follow the above 
guidelines. 





oubmit a Paper | 


Figure 4. General Guidance Web Page for ISCAS ’98 Paper Submission 


Maj 





Primary Author or Contact Author Identification 


| | 


Last Name (only) eMail Address 





Is this a Resubmission? © No € Yes: Paper Serial Number. | 


Topic:| | ubT opic: | 


Additional Comments 





Submit Paper Information Clear Form | 


Figure 5. Initial Paper Submission Application - POC Author / Paper Information 


Form 
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receiving directory. The Perl CGI then informs the Tango CGI that a paper has been 
successfully uploaded to the server by sending the received paper’s tracking number. The 
Tango CGI updates the database to indicate that the paper was received and then sends a 
successful upload message to the client’s browser which indicates that the paper, submission 
process has been completed. The message also displays all of the associated paper and 
author information from the Access database which was submitted using the forms in 
Figures 5 and 6. The flow chart in Figure 8 provides a graphical overview of the initial 


submission application. 


es a 


‘Title of _ First Name MI Last Name IEEE Member? y 


| eMail Address: | 


Cm, Decade 


Home Phone Numb er Work Phone Number FAX Number | 
















Department Street {POBox 


as TEP Sees 


City _- State/Province = Mail / Zip Code Country 








Additional Authors: Is there another author? “ Yes © No 


See Eee 


Last Name (only) eMail Address 





submit Author information | Restore Original Values 


Figure 6. Initial Paper Submission Application - Author Personal Information Form 


Pog) 


Paper Submission Form 





Please complete the paper submission process as described below. 


First, press the "Browse..." button to select the “postscript” or "pdf file contaiming a surnmary of the 
paper to be submitted. (You may have to pull down the “Files of type:" menu to select your file) 

Note: If you are using Intemet Explorer, this may not work properly. If this is the case, please find a 
copy of Netscape Navigator to use for this upload. 


File to upload: Browse... | 


Next, Press Here to upload the file! 
(This could take several mutes depending on connection speed and file size.) 


Finally, this paper has been assigned a tracking number as follows: 
Tracking Number: 1909 
Please write this number down for future reference. Should you heve problems submitting your paper, 


you will need this number along with the last name and email adress of one of the authors just entered 
in order to attempt a resubmission. Always use this number when referencing your paper. 





Figure 7. Initial / Final Paper Submission Application - Paper Submission Form 
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Figure 8. Initial Submission Application Flow 
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In order to resubmit a paper, an author would start with the form in Figure 5. 
By entering the POC author’s “Last Name” and “Email Address”, and by selecting “Yes” 
for the question “Is this a resubmission?” and including the assigned “Paper Serial 
Number”, a paper could be resubmitted (uploaded again). The resubmission would use the 
same paper number and would be named with an “r”’ appended to the paper number. This 
particular approach turned out to be somewhat dysfunctional and will be discussed further 
in the next chapter. 

When the deadline for submitting papers is reached, the security level of the - 


paper submission application must be increased to prevent any further general public 
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access. At this point, only administrative personnel with login access will be able to work 
with the submission application. This will eliminate problems associated with late 
submissions by simply disallowing them. The capability to completely automate this 
deadline action exists, although it was not integrated into the ISCAS ‘98 submission 


application. 
c The Login Application 


The paper submission application is the only application which was made 
publicly available over the Internet. This was essential in order to allow all potential 
authors the opportunity to submit papers for consideration. The remainder of the 
prototype’s applications, however, would require login protection in order to prevent 
unauthorized access. Each of the remaining applications were assigned a security level 
which corresponded to the access level of those who could use it. Users with login access 
were assigned an access level which determined which applications they could use. The 
login application was originally available from the left hand menu section of the ISCAS °98 
home page shown in Figure 3. The link was removed after the Selection Phase was 
completed and is not shown in the figure. Clicking on the link would start the login process 


by displaying the form shown in Figure 9. 
Please Log In 


Enter your User Name and Password to gain access to the system. 


User Name: = 
Password: LL _ 
Login Reset | 


Figure 9. Login Application - Access Form 


A successful login will result in a welcome screen as shown in Figure 10 
which will list only those applications for which a user has access. The login application 


cannot be bypassed by simply bookmarking applications in the browser. This is because 
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each application validates the access level of the user when started. A user’s access level is 
stored on their machine in a browser “cookie” when they use the login application. Existing 
cookies cannot be saved for future use because they will expire if there is no interaction 


with the ISCAS prototype for more than 15 minutes. 


d. The Acknowledgment Application 


The paper submissions require validation to ensure that they have been 
received in good condition (non-corrupted) and that they have been converted into PDF 
files. On-line readability of the electronic papers is the primary concern (not content). 
After a received paper. has been decompressed, converted into PDF, and moved into the 
designated on-line directory, it can be confirmed and acknowledged by sending an e-mail 
message to the POC author. Validation is conducted by administrative personnel with login 
access to the acknowledgment application. When the acknowledgment application is 
selected from the welcome screen in Figure 10 (link to application is not listed in the 
sample shown), the browser displays a listing of papers which need to be acknowledged. A 
sample listing is shown in Figure 11. There is a button (which is not shown) at the bottom 
of the paper listing which will display the next 100 submitted papers when pressed. A 
paper can be confirmed by clicking on its listed title which will open the paper in the 
browser using Adobe Acrobat Reader and its associated web browser plug-in. 

After confirming the condition of the paper (by reading it), a confirmation 
form can be opened by clicking on the desired paper’s number in the “Paper” column. 
Figure 12 displays a sample confirmation form which opens up with the paper and author 
information already filled in along with the message to be sent to the POC author. The 
sample shown is the form for confirming receipt of a paper in good condition. Beneath this 
form in the browser window is another form (not shown) which allows for notification that 
a failed transmission or other problem has occurred which will require the POC author to 
resubmit the paper. The message can be edited in either form to provide more detailed 


information if needed. Pressing the “Send Confirmation” button will send an e-mail with 
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Welcome, Jim Webmaster! 
Last login: 01/16/1998 19:33:22 


Paper Reviews 


2 
® Locate Paper Reviews For Final Status Selection 


Paper Browsing & Scheduling 
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Figure 10. Login Application - Welcome Screen / Application Menu 
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Confirm Paper, Then Send Acknowledgement 
Click on Title to Confirm (View) Paper. 

Click on Paper Number to Acknowledge Paper. 

Use Caution: Click on the Topic to edit the paper record. 


There are 150 matching records. Displaying matches 1 through 100. 
[Paper os [Topic |Title 


= ibe none is Statistical Analysis of Chaatic 
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Figure 11. Acknowledgment Application - senate List of Paper Submissions — 
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ISCAS 98 Paper Cenfirmation Form 


The following form confirms that ISCAS 98 has a readable electronic version of the paper. 
You may modify the message, if desired. 


Acknowledgement of Paper Submission for IScas9s 
Paper Number: 1804 
Paper Topic: 1.12 


Title: Statistical Analysis of Chaotic Communication Schemes 


Authors: Abel, Schwarz, Goetz, 


The above referenced paper has been received in good condition by the 
ISCasSs8 Secretariate. Your submission will now be sent for review. You 
Will be notified of the outcome of the review around December 15, 1997. 
This year the author's kit will be sent to you in electronic form. The 
list of accepted papers will also be posted on the symposium website. 
Please reference the above paper number in any future correspondence. 


Thanks for submitting your work to ISCasg9s. 


Send Confirmation |  PesetConfirmation 


Watch the botiom of your screen after pressing "Send Confirmation". 
ait for the "Done" message, then press this button: 


Ack Paper Record i 





Figure 12. Acknowledgment Application - Paper Confirmation Form 
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the message to the POC author. After sending the message, pressing the “Ack Paper 
Record” button will tag the paper record in the database as having been acknowledged and 
will return the user to the listing in Figure 11, which will no longer have an entry for the 
paper just acknowledged. In the form not shown, the buttons are titled “Send Negative 
Acknowledgment” and “Nack Paper Record” respectively. They work the aig except the 
database record is tagged to indicate that the paper has not yet been received in good 
condition. 


2: The Selection Phase 


a. Background 


This phase starts immediately after the deadline for paper submissions has 
passed. Using traditional methods, copies of the papers would be mailed out to the various 
reviewers for an acceptance recommendation. This would require anywhere from three to 
five mailings per paper. Included with each copy would be a review form to allow the 
assigned reviewer to evaluate the paper and to make an acceptance recommendation to the 
Technical Program Commuter. Each reviewer would read the paper and fill out the 
attached form. The completed review forms would then be mailed back to the TPC. The 
TPC would consider all reviews submitted for each paper in order to make a final 
acceptance determination. The accepted papers would then be arranged by subtopics into 
related groups. Borderline (maybe) papers would be used to fill up any groups with empty 
positions. The groups of accepted papers would then be scheduled into presentation 
sessions for the upcoming conference. 

Electronic distribution of papers has previously been attempted using an 
FTP server at other conferences; however, this method requires the reviewers to locate and 
transfer the papers to their machines and then to print them out for review. Also, FTP 
servers do not provide any automated tracking or management services, but they are subject 
to the same networking issues as a Web server. Submission of review forms has also been 


allowed via e-mail, but this requires manual sorting for analysis by the TPC. An example 
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of an FTP server with e-mail review forms is that used by the IEEE INFOCOM ’98 
Conference on Computer Communications.[Ref. 1] 

With regard to the ISCAS conference, the functional requirements of the 
selection phase were readily adapted to the on-line prototype. The processes for distributing 
papers and collecting review forms were completely automated on-line and required no 
postal mailings. Both the TPC paper selections and the scheduling of presentations were 
also automated on-line. However, grouping the accepted papers by topic and session prior 
to scheduling proved too complex for an automated application due to the difficulty in 
constraining the process. There were over 900 accepted papers with more than 50 topics 
and 180 sessions to deal with. The TPC held a two-day meeting to resolve this manually. 
All other selection phase functions could be performed using a standard web browser over 


the Internet. 


b. The Paper Review Application 


After all of the papers have been received and acknowledged, the TPC 
selects the paper reviewers and assigns a list of paper numbers to each reviewer. There 
were over 200 reviewers for ISCAS °98. Each reviewer was assigned multiple papers to 
review; and each paper was assigned to be reviewed by at least two reviewers. Well over 
2000 reviews were expected for the 1200 plus papers to be reviewed. A special multi-user 
login account was created for use by all paper reviewers which would give each of them 
access to the on-line review application. The account information was distributed by e-mail 
to the reviewers via the TPC along with their assigned paper numbers. A reviewer could 
then use the login application from Figure 9 to gain access to the paper review application. 
Clicking on the link titled “Review a Paper” near the top of the welcome screen (shown in 
Figure 10) would launch the paper review application. Figure 13 displays the form used to 
find a paper for review. After a reviewer entered an assigned paper number and pressed the 
“Find Paper” button, the review form would be displayed in the browser for that paper. The 


review form is shown in three parts: Figure 14, Figure 15 and Figure 16. 
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Figure 13. Paper Review Application - Access Form 
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Figure 14. Paver Review ition.” Top of Review Form: Paper and ys 
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4. Evaluation of the paper: 

Excellent Good Far Poor 
1. Onigmnality (novelty) of the work reported: _ Ca #° { Scat 2c 
2. Significance of the topic addressed by the paper: 7 j , *) Ok ge 
3. Clarity of presentation / readability of the paper: m4 Co Ci C4 


4. Technical correctness of the work reported: ai (2 C3 Cc 4 
5. Interest and relevance to ISCAS audience: CF 2 c 22 C4 
6. Reference to previous work in the literature: i C2 @*: C4 
7. Adequacy of the page length of the summary: Ca 2 3 C4 
&. Overall quality of the paper: | 7 co rc 2? C3 + 4 


9. Summary recommendation: | cr | 


B. lf you reviewed N papers, please rank this paper as n of N. 
Relative ranking of the paper: o out of | papers 


C.(Optional) Ifthe paper does not fit m the subtopic as currently categonzed, please suggest 
amore appropriate subtopic: 
Subtopic: 





Figure 15. Paper Review Application - Middle of Review Form: Criteria and Ranking 
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D.(Optional Please write here any comments you may have for the author: 





E. (Optional) Confidential comments to the TPC (Comments below will be withheld from the 
authors): 


F. Reviewer Data: 


Reviewer's Name: ee 
Reviewer's Affiliation: | 


Reset Values 





Save 


Figure 16. Paper Review Application - Bottom of Review Form: Comments / Reviewer 


ID 
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The top of the review form, Figure 14, displays the instructions, the paper 
information from the database, and a list of the authors. Most importantly, the paper title is 
a link to the actual on-line version of the paper. The reviewer can click the link to either 
open the paper in the browser or to download the paper for off-line reading. After reading 
the paper, the reviewer can come back to the review form to complete the evaluation fields 
shown in Figures 15 and 16. The “Recommendation” field contains a drop-down menu 
which has three options: “Accept’, “Not Sure’, and “Reject”. Pressing the “Save” button 
submits the review form, which is then stored in the Microsoft Access database. The 
reviewer is then automatically returned to Figure 13 to allow review of another paper. Once 
a review is submitted, it 1s not accessible by the reviewer or anyone other than a TPC 
member. Submitted reviews cannot be modified by anyone. If a reviewer wishes to change 
the evaluation of a paper from a previous review, a new review form can be submitted. The 
TPC member who makes the final decision will be able to see that both reviews came from 


the same reviewer and can go with the information from the latest review. 


Cc. The Paper Selection Application 


| All previously submitted reviews are only accessible via the paper selection 
application which also requires login access. Only TPC members have access to this 
application, which has a higher security level than the paper review application. Before this 
application was made available to the TPC, approximately 50 paper records had to be 
tagged for “special” access to prevent TPC members from being able to access the reviews 
of their own papers or those of their assigned reviewers. This was required since over 50 of 
the reviewers and TPC members had also submitted papers for the conference. These 
papers could only be approved or rejected by the TPC Chair, although assigned reviewers 
could still review them. 
Clicking on the link titled “Locate Paper Reviews For Final Status 
Selection” shown in Figure 10 would launch the paper selection application and bring up 
the search form shown in Figure 17, which contained two drop-down menus for locating the 


reviews. The “SubTopic” menu allowed for selection of reviews by any or all of the 
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Use this form to locate all reviews for all papers in a desired SubTopic which 
meet the selected Final Status. 


Locate Reviews by SubTopic: & All SubTopics | 
and Final Status: [Pending | 
Find Reset Values | 


Figure 17. Paper Selection Application - Review Search Form 


subtopics originally listed in the “First Call For Papers”. The “Final Status” menu allowed 
for selection based on whether a paper had been “Accepted”, “Rejected”, or was still 
“Pending”. Additional options were available in the “Final Status” menu for the TPC 
Chair, which would allow access to the “special” papers. Pressing the “Find” button would 
bring up a list of the submitted reviews having the desired subtopic and final status. 

Figure 18 provides a sample listing of paper reviews located by the paper 
selection application. The detailed review information is accessible by clicking on a 
review’s “Date Submitted” which would bring up the form shown in Figure 19. A final 
selection decision for the paper, or “Final Review Status’, could be made from this form by 
using the drop-down menu provided and then pressing the “Save Final Status” button. This 
would return the TPC member to the review listing in Figure 18, which would no longer 
have any reviews listed for that paper. 

Near the end of the paper selection process, the TPC members held a oo 
day meeting to group the accepted papers into related sessions. One of the goals for the 
meeting was to identify sessions which were not quite full in order to fill them with papers 
that had been marked as “Not Sure” by reviewers. There were 180 sessions to cover the 
three day period of the conference. The TPC members broke the papers down by subtopic 
and then by session. New subtopics were added to accommodate papers which did not fit 
well within any of the existing subtopics. This process involved manually laying out the 


sessions by placing yellow “Post-it” notes across an entire wall to ensure that no related 
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sessions would be scheduled simultaneously. At the end of this meeting, a final decision 
had been made for all submitted papers and the accepted papers were ready to be scheduled 


and sequenced into a presentation order within the identified sessions. 


d. The Presentation Scheduling Application 


Each of the conference sessions was available via the presentation 
scheduling application, which was also only available to the TPC members. The 
application would launch by clicking on the link titled “Schedule Papers into Sessions” 
shown in Figure 10. It started with the form shown in Figure 20. The three drop-down 


Use the drop-down lists below to choose a 
session in which to schedule presentations. 


Day AM/PM Session Number | 


Session: Monday ~| [Morning A | ~| 
ein Sesion | 


Figure 20. Paper Scheduling Application - Session Access Form 


menus shown could be used to select a particular session in which to schedule papers for 
presentation. Pressing the “Display Session” button would bring up a form like the sample 
shown in Figure 21. If the “Session Title”, “Session Type”, “Chair’, and “Co-Chair” fields 
were not already filled in, a simple form (not shown) for labeling the session would appear 
first. Papers already scheduled into the session would be listed in order of presentation as 
shown. Papers could be added by simply entering a “Paper Number” and the desired 
“Order” of presentation and then pressing the “Add Paper” button. The rest of the- 
information shown for a paper would be automatically extracted from the database and the 
list of papers would be updated and sorted. Attempting to add a paper which had already 
been scheduled in this session or any other session would generate an error message to 
notify the user that the paper could not be added without first removing it from its other 


scheduled session. Clicking on a paper’s “Title” would bring up the form shown in Figure 
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22, which allowed for removing the paper from the session or changing its presentation 
order. Pressing either of the two buttons on the form would make the desired changes and 
return the user to the form in Figure 21 to display the updated list of presentations for the 
session. After all the papers had been scheduled, the Microsoft Access Database could be 
queried to automatically generate the ISCAS conference schedule. 


Click on a paper title to allow removal or to change the paper’s presentation "Order". 
The presentation “Mode” is the first choice listed by the author. 


Session Title: Parameter Estimation 
Session Type: Lecture 
Chair: Professor Eugene I. Plothin, Concordia unrversity, Canada 
Ce-Chair: 


Session: Monday MomungA Session 1 





Add a paper to this session: 


Paper Number: — Order. J Add Paper | 


Figure 21. Paper Scheduling Application - Session Scheduling Form 
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Use this form to change the order in which this 
paper is presented during the session or to 
remove this paper from the session. 


Day: Monday 
AM/PM: Moming A 
Session: 1 


Presentation Order: i i 


Paper Nwmber: 584 


Topic: 62 

Status: Accepted 

Presentation Mode: Lecture 

Paper Title: On the Harmonic Analysis of Speech 


Change Presentation Order | 
Remove from this Session. | 


Figure 22. Paper Scheduling Application - Change Form 


au The Final Submission Phase 


a. Background 


This phase is very similar to the initial submission phase. The primary 
difference is that instead of mass distributing a “First Call For Papers’, the authors who 
submitted papers will be notified as to whether their papers have been accepted or rejected. 
This requires administrative personnel to mail out formal letters of notification to each POC 
author. Those authors with accepted papers are directed to submit final, camera-ready 
versions of their papers for publishing. The submission and acknowledgment functions are 
essentially identical to those in the initial submission phase. 

For the ISCAS prototype, a very simple query application was created to 
pull and format paper and author information from the database to be merged with a formal 
notification letter in a word processor. There were two different letters, one for accepted 


papers and one for rejected papers. A temporary employee was hired to manually merge 
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and mail all of the letters. With regard to the submission and acknowledgment functions, a 
new suite of applications was developed using a new database. Many lessons learned from 
the initial submission phase were incorporated into the new applications in an attempt to 
minimize or eliminate most of the problems experienced with the use of the original 
applications. The specific problems and lessons learned will be discussed in detail 
throughout the next chapter. The acknowledgment application remained relatively 


unchanged, but the paper submission application was completely overhauled. 


b. The Final Paper Submission Application 


The link from Figure 3, titled “Click Here to Submit Final Papers!”’, displays 
the new application links shown in Figure 23. There is a link for accessing the login 
application and two other links which now more clearly differentiate between the 
submission and resubmission aspects of the process. Clicking the “Submit a Final Paper” 


link brings up the form shown in two parts, Figures 24 and 25. The first thing asked for in 


ISCAS '98 Final Paper Registry 


For Accepted Authors: 


© Submit A Final F 
® Resubmit A Final Paper / Edit Paper and Author Date 


For Administrative Personnel: 
® Administrative Login 
Figure 23. Final Paper Submission Application - Access Page 


Figure 24 is the previously “Assigned Paper Tracking Number” which is required to later 
verify a paper submission as an accepted paper. The author information is being requested 
again due to previous problems in order to ensure that the information will be correct when 


published in the conference proceedings. The second part of the form, in Figure 25, 
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Paper / POC Author Information 


Enter Your Assigned Paper Tracking Number: 

Please enter the following mformation for only one author. This should be the author who will be 
acting as the primary Point of Contact (POC). You will be able to add more authors on the next form. 
Carefully check the email address, fax and mailing label elds to ensure proper receipt notification. 
The mailing label should contain a complete address Gnchiding name and affiliation) exactly as it 
would eppeer on an exrvelope addressed to this author. The phone numbers are optional. This 


information must be correct. It will be used in the final publishing of your paper and will not be 
edited by the conference secretariate. 


Title: [a a * other |  #8€=)3=— 

Last Name: a Sr 
First & eee eee ae 
Middle: 

Email 

Address: 

Fax: , a Ses eee | 
Werk <<< a | 
Phone: 

Home Ke ee 
Phone: 

Affiliation: , rr 
tek no “other Country_| 





a 


Figure 24. Final Paper Submission Application - Top of Paper/Author Information 


Form 
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Please enter the following paper information. Do not use all upper case for the title and abstract. This 
information will be sent directly to the publisher for accepted papers, so please check grammar, 
format and spelling carefully. 

Paper an 
Title: 


Abstract: ig — 


nae 


® Each of the contributed papers for the ISCAS'98 conference has to be presented by “one” of the 
authors. Please enter the name of the presenter in the Comments field below. 

® Please also enter a brief summary of biographical information for the presenter in order for the 
session chair to introduce him/her. 

® Eech conference room will be equipped with overhead and slide projectors. If the presenter has 
any additional audio/visual needs, please enter them also. 


Conunenis: 





eeemorn | Required for future access & resubmissions!) 
Clear Fields | Continue... | 
Figure 25. Final Paper Submission Application - Bottom of Paper/Author Information 


Form 
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collects the “Paper Title” and “Abstract” for publishing. There is also a comments field and 
a “Password” field which is required to allow the author to have future access to the paper 
record. Pressing the “Continue...” button submits the data for storage in the database and 


displays the paper record as shown in Figure 26. 


Accurate CMOS Switched-Current Divider Circuits 


«** Paper Number: 62 *** 
Point of Contact (POC Author) desires notification by: Email 


Abstract: This paper presents a highly accurate current divider using switched-current GD) 

technique. The circuit accurately divides an mput current by two with 3 cycles at each iteration. The 
accuracy of the division increases as the number of iterations increases. In practice, however, the | 
accuracy is imited due to the clock-feedthrough errors. The issue of accuracy limitation is addressed. 


The extension to array structures for low-power/low-voltage A/D and D/A converter circuit designs 
is also discussed. 


Connsexts: 


Edit The Above Parameters... | 


Press the "Edit The Above Parameters..." button to change the title, notdication method, abstract or 
comments. 





Click an Author's Name te edit his/her record. 
Nete: the POC author record cannot be deleted. 


Bi crea edu 517-353-1980 Michgan State 
University 
co Mr Jin Sheng Wang jwengjins Michigan State USA 
University : 
Change POC | Add an Author... | 

|Submission Attempt [Type [Date Submitied Received? |Date Received | Confirmation | 
| 1 [Final lo1sa9ri998 14:43:15 | Yes (01/29/1998 14:43:49 | Pending 

st Resubmit ie Beets | 


Figure 26. Final pas Submission Application - Sample Paper Record 


‘dl 
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From within the paper record, an author now has complete control over the 
information stored in the database for his/her paper. The “Edit The Above Parameters...” 
button allows modification of the paper information in a form very similar to the one shown 
in Figure 25. The only difference being an added drop-down menu allowing selection of a 
POC notification method for acknowledgments. The choices are “Email” (default), “FAX’’, 
and “Postal Mail”. Clicking on an “Author Name” allows editing and/or deleting of the 
information for that author using a form nearly identical to the one in Figure 24 (minus the 
paper tracking number field). Additional authors can be added by pressing the “Add an 
Author...” button which also uses a form like that in Figure 24. A new POC author can be 
designated by clicking a radio button under “POC” and pressing the “Change POC” button. 
Pressing the “Submit/Resubmit the Paper...” button will bring up a form like the one shown 
previously in Figure 7. Submitting the paper with that form will return the user to Figure 
26, which will display a sequential list of submission attempts near the bottom of the form. 

After administrative personnel have confirmed the submissions with the 
acknowledgment application, a new button will appear on the form in Figure 26 which will 
allow authors to access and read their own papers on-line. Authors can use this option to 
see exactly how their papers will appear when published. To regain access to their paper 
records for viewing, resubmitting and editing, authors would click on the link titled 
“Resubmit A Final Paper / Edit Paper and Author Data” in Figure 23. This would bring up 
the form in Figure 27, which requires a “Paper Number” and a “Password” to open the 
record to the form shown in Figure 26. The flow chart shown in Figure 28 provides a 
graphical overview of the final submission application. 


4. The Publishing Phase 


This phase consists of the conference registration process in addition to the 
publishing of accepted papers in the conference proceedings. These two functions are 
closely related because each of the accepted papers requires registration by at least one 


author. The registration is to include payment for the conference and any additional paper 
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Enter your paper number and password to open the paper record. 


Once the paper record is opened you may: 

@ Edit the paper record 

®@ Submit (or resubmit) the paper 

®@ View the paper online (several days after submission) - Requires Adobe Acrobat Reader Plugin 
@ Eds author records 

® Add authors 

® Delete authors (POC author cannot be deleted) 

@ Change POC author designation 


Paper Number: | 
Password: | 
Find oo J 


: 27. Final Paper Submission Application - Resubmission/Editing Access Form 
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Display Figures 


Figures "| 24 & 25 cue 


Paper & 
Tango CGI POC Author Tango CGI 


Data 
Figure 23 Siew 


Submit / Display 


Resubmit Figure 23 
Figure 26 
, eure “| Change POC Author 


Submission ID, Store 


Paper ; Data 
Add / Edit 


ee a4 Author Tango CGI 
Store ieee Form 
Paper Similar to 
Form 


Figure 24 


Perl CGI Similar to 
Figure 25 





Figure 28. Final Submission Application Flow 
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publishing costs. Ideally, a paper would not be published if registration and payment for an 
accepted paper were not received prior to the publishing deadline. 

One of the original goals for this prototype was to include an on-line registration 
application which would be linked directly with the database of accepted papers. This 
would have provided an automatic integration of the registration and publishing efforts. 
Time constraints and security concerns for accepting on-line credit card information 
precluded the development of a registration application; however, an on-line registration 
capability was provided through an outsourcing contract with a commercial provider. This 
meant that the two processes would eventually require manual integration. Furthermore, 
the actual publishing of the papers would, unfortunately, not commence in time to be 
discussed further in this document. Therefore, prototyping efforts for the functions 
associated with the publishing phase will be left for future research. 

Although only the initial submission and selection phases have been fully 
completed at this time, the overall prototyping effort has been extremely successful in 
automating the majority of the ISCAS conference planning functions over the Internet. In 
the next chapter, specific performance of the prototype will be analyzed and lessons learned 


will be discussed for those phases which have been completed. 
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IV. PROTOTYPE ANALYSIS, LESSONS LEARNED AND 
RECOMMENDATIONS 


This chapter discusses the results and lessons learned from the various phases of the 
ISCAS prototyping effort. Recommendations for overcoming most of the described 
problems are included. In addition, the final section describes an idealized set of procedures 
for providing fully automated submission and acknowledgment capabilities in support of an 


on-line conference planning effort. 


A. OVERVIEW OF RESULTS 


This section provides an overview of results which could be identified Aging time 
this document was prepared. All four phases of the prototype are listed; however, no results 
were available for the final submission and publishing phases. The initial submission and 
selection phases had been fully completed; therefore, specific results could be identified and 
analysis provided in their respective sub-sections. 


We Initial Submission Phase 


Over 950 unique papers (approximately 80% of the total) were submitted for review 
by authors using the on-line system. Many of the authors for the on-line submissions also 
mailed in paper copies. Another 250 papers were received by postal mail only. Authors 
from 62 countries participated. Approximately 98% of all paper acknowledgments were 
sent to the POC authors via e-mail by samiieeanve personnel using the Tango CGI 
acknowledgment application. The remaining acknowledgments had to be faxed or mailed 
due to missing or invalid POC e-mail addresses. 

Approximately 30% (555 of 1759) of all paper records in the database were 
determined to be duplicates which resulted in just over 1200 valid paper records out of the 
1759 in the database. This was the largest single problem faced by administrative personnel 
and lead to longer than expected delays in sending acknowledgments to the authors. Some 


of the duplicates can be attributed to growing pains with the new on-line methodology. In 
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particular, many authors failed in their initial attempts and repeated the submission process 
until they either succeeded or gave up and mailed their papers. Each on-line attempt created 
a new record in the database. In many cases, the mailed papers resulted in another duplicate 
record for those papers. Although a paper resubmission option was available, authors often 
overlooked or misunderstood how to use it, resulting in limited success and adding to the 
duplicate record problem. 

Many authors were not completely comfortable with the on-line process and, 
therefore, mailed in copies of their papers in addition to their on-line submissions. These 
added to the problem in cases where an assigned paper tracking number had not been 
included with the mailed copy. To further aggravate the problem, the delays in 
acknowledging the papers resulted in even more mailed submissions as authors wanted to 
take all precautions on their part to ensure that their papers were received. The lessons 
learned section below will discuss recommendations for avoiding this duplicate record 
problem. 


De Selection Phase . 


Well over 2000 paper reviews (100%) were conducted using the on-line paper 
review application. All of the accepted papers were selected and scheduled into pre- 
designated conference sessions using the appropriate Web applications. No papers or 
review forms had to be copied or mailed to distribute them to the reviewers. All reviewers 
accessed the papers and submitted the reviews using the on-line applications after receiving 
an e-mail which listed the papers assigned for their review. Other than a handful of 
reviewers having browser configuration difficulties on their client platforms, the 
applications built for the review and selection of papers worked as designed with no major 
difficulties. The selection phase was the most successful in terms of automating the 
underlying processes and collaborating over the Internet to plan and coordinate for the 
conference. As a result, there are no significant recommendations for functional changes to 


the selection phase in the lessons learned section below; however, the manual method used 
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to group and schedule the sessions for the conference deserves consideration for future 
research and analysis. 


3. Final Submission Phase 


Approximately 950 papers are expected to be submitted in camera-ready format 
during the final submission phase, which is almost identical to the initial submission phase 
with regard to the process requirements. Consequently, many of the lessons learned from 
the initial submission phase resulted in significant modifications to the on-line application 
which would be used for final paper submissions. As of this writing, no results from the 
final submission process were available; however, preliminary indications are that the new 
submission application is working as designed. Thus, no additional lessons learned, beyond 
those from the initial submission phase, will be discussed below. The last section in this 
chapter does, however, propose additional requirements which could significantly impact 
this phase as well as the initial submission phase for a future conference. 


4, Publishing Phase 


This phase had not yet been reached at the time this document was prepared; 
however, the on-line registration capability had already been established through an 
outsourcing agreement with a commercial provider. No details or results were available 
regarding the registration efforts. Additional efforts to explore the processes involved in the 


publishing phase will be left for future research. 


B. LESSONS LEARNED 


The major problems encountered during the ISCAS prototyping effort can be 
broken down into three general areas: excessive duplication of paper records in the 
database, submission of corrupt electronic files, and communication problems with POC 
authors. This section discusses the lessons learned in dealing with these problems, most of 
which came from the paper submission and acknowledgment processes during the initial 


submission phase. 


Sey 


1. How to Avoid Duplicate Paper Records in the Database 


There were several different identifiable causes which resulted in the problem with 
duplicate paper records. This sub-section discusses the causes and provides a recommended 


approach to overcoming each of them. 


a. Allow Multiple Submissions for each Paper Record 


The initial submission application provided limited support for allowing 
authors to resubmit their papers. Only one paper submission was originally supported for 
each paper record. The unique database index assigned for each paper record was used as 
the paper tracking number; therefore, each paper received was named using its database 
index. For example, paper number 27 was received and stored in the system under the 
name “27.pdf’. To support a resubmission option, the file upload CGI was modified to 
name a resubmitted file as “27r.pdf”. Under this system, only two submissions were fully 
supported for each paper; however, in many cases, more than two submissions were 
attempted, which resulted in either overwriting previous data or the addition of a duplicate 
record. 

The recommended solution to this problem is to modify the submission 
application and the underlying database to allow multiple submissions per paper record. 
This could be accomplished by using a separate submission tracking number to uniquely 
identify each submission. For example, paper number 27 might have three submission 
attempts which have been numbered 43, 54 and 67. The database record for paper number 
27 would store these three numbers to provide links to the submitted papers. This 
methodology was implemented in the final paper submission application and preliminary 


results indicate that it is working as proposed. 


b. Provide a Distinct Resubmission Option 


Another problem with the initial submission application was its integrated 
resubmission option. Authors who wished to resubmit had to use the same application and 


had to re-enter much of the previously submitted data along with their paper tracking 
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number. Additionally, a radio button designating the attempt as a resubmission had to be 
selected to prevent creation of a duplicate record. Unfortunately, many authors simply 
overlooked the button, which was not very prominent on the form. I[n any event, it wasn’t 
very clear that authors needed to return to the submission application to resubmit. 

Providing a separate resubmission function (with its own link) would more 
clearly indicate how the resubmission process works. In addition, allowing each author to 
use their assigned paper number in combination with a personal password to access the 
resubmission application would ensure that each resubmission was attached to the correct 
paper without requiring all previously entered paper information to be re-entered. To 
support this, each author would be allowed to enter a password of their choice during the 
initial submission process. This password would be stored within the paper record and 
would be used to validate any attempt to access the paper record. This recommended 


feature was also included in the final submission application. 


C. Allow Paper Record Editing by POC Authors 


The initial submission application provided no support for allowing authors 
to recall and/or edit previously submitted paper and author information. One of the initial 
indicators that this was a problem occurred during initial author interaction with the 
submission application. Many authors wanted to test the process by running through the 
submission application once before actually using it. Each test created a useless paper 
record since the authors could not go back and edit the record. Also, some ator would 
use their browser’s “Back” button to go back to a previous form in order to make changes to 
submitted information; however, this simply resulted in a new record with the updated 
information. Further, when authors wanted to change paper or author information from 
what was previously submitted, they would go through the entire submission process, 
which would create a duplicate paper record each time. 

To overcome these problems, authors would need the capability to edit 
previously submitted information. The same password and paper number access capability 


mentioned above could be used to allow authors to edit their paper records. Once the paper 
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records were accessed via password, an on-line update form could be used to edit the paper 
and author data. Additionally, a hidden time-stamp could be embedded within the on-line 
forms to prevent inadvertent duplication through use of the back button. The Tango CGI 
could use the time-stamp to check for a pre-existing record which could be updated rather 
than always creating a new one. These editing features and techniques were included in the 


final paper submission application. 


d. Provide Extensive Paper Search Capabilities 


The administrative applications originally provided to support the 
processing of papers received by postal mail did not fully support those papers which had 
already been submitted on-line. Ideally, any paper, which had been submitted on-line 
before mailing it, would not be submitted again (creating a duplicate record) by 
administrative personnel when received. A query capability would be essential in aiding 
administrators to find pre-existing paper records. The minimal functions which were 
provided only included the capability to search by paper number or by author’s last name. 
These were not sufficient for two reasons. First, authors seldom included their paper 
numbers, which were assigned on-line, with the mailed versions. And second, a search by 
author’s last name could result in a lengthy list of records which required administrators to 
open the records one at a time to compare paper titles. 

To provide better administrative support for mailed papers, enhanced search 
capabilities would be required. The capabilities to search by paper number, by author’s last 
name, and by paper title would all be essential. A sub-string search capability could also be 
included with the “by author” and “by title” options. Each search would need to display a 
list of found papers which would include paper numbers and titles for rapid correlation. 
The search by author function would also need to list the authors’ last names. Additionally, 
the search functions could be made more easily accessible by including them all on the 
same Web form. These enhanced search capabilities were also included with the final paper 


submission application. 
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é. Provide Rapid Feedback to POC Authors 


Untimely feedback was probably the most significant contributor to the 
problem with multiple submissions which resulted in duplicate records. The lack of timely 
feedback manifested itself in a variety of ways. First, when authors submitted large files via 
the on-line Web application, no feedback was provided after pressing the Submit button 
until after the file fad been completely transferred to the server. Due to the extended delay, 
many authors felt their might be a problem and would resubmit again and again, creating a 
new record with each attempt. Second, all but about 100 of the papers submitted on-line 
were submitted during the final five days before the deadline. This created additional file 
transmission delays due to the heavy server load which resulted from having the combined 
Web, database and file upload services all being provided from a single machine (the NT 
Server). And finally, no formal acknowledgments were sent to any authors until well after 
the deadline for submissions had passed. This resulted in additional resubmissions as 
authors wanted to make sure that their papers were received. 

To overcome delays and provide better feedback, several steps could be 
taken. First, the Perl script CGI could be modified to provide a dynamic indication that file 
upload activity is progressing. It is believed that this can be done; however, a preliminary 
attempt to modify the Perl script proved unsuccessful. Second, the server activities could 
be distributed across multiple platforms to minimize server load during peak access times. 
For example, the Web server, the database server and the file upload application could all 
be supported by separate platforms. The file upload CGI was ported over to a high capacity 
SGI server for the final submission phase while the Web and database services remained on 
the NT server. Additionally, a script could be generated to automatically email a 
preliminary acknowledgment to the POC author immediately after each submission. 
Although this capability was not provided in the final submission application, it could serve 
to reassure authors that their papers had been received and were awaiting verification (at a 
later date) by administrative personnel. Finally, administrative personnel could 


acknowledge submissions as they arrived rather than after the deadline had passed. 
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Unfortunately, personnel resources simply were not available to provide immediate 
acknowledgments during the final submission phase. 


2: How to Avoid Corrupt Paper Submissions 


Corrupt files created additional difficulties for administrative personnel during the 
confirmation and acknowledgment of papers received over the Internet. This sub-section 


discusses how to deal with the file corruption problem. 


a. Provide On-line Guidance and Support 


Approximately 50 of the 950 papers submitted on-line had some form of 
font corruption. Almost all of these problems were directly attributable to embedded two- 
byte fonts which are typical in most foreign language software packages. Received 
Postscript files containing two-byte foreign fonts could not be converted into PDF files. 
PDF files received with the two-byte fonts could not be viewed on-line using the English 
version of Acrobat Reader. In all such cases, administrative personnel had to work closely 
with the authors to coordinate removal of the two-byte fonts and resubmission of the papers. 

Extensive on-line support could help to prevent the foreign font problem in 
the future. This support could include explicit guidance, examples of on-line PDF files, and 
links to English versions of the essential software, like Acrobat Reader. Authors could also 
be directed to Adobe’s Web site for white paper discussions on specific technical issues. 
Finally, an on-line list of frequently asked questions, or FAQ, could be maintained to aid in 
limiting and/or preventing further corrupt submission problems. 


3. How to Avoid Problems Contacting POC Authors 


The ability to contact POC authors for sending acknowledgments and notifications 
was another area of concern with the prototyping effort. This sub-section provides some 


recommendations to minimize these communication problems. 


a. Allow POC Authors to Choose a Desired Contact Method 


During the acknowledgment portion of the initial submission phase, 


administrative personnel were unable to send e-mail acknowledgments to the POC authors 
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for every paper record. Approximately 40 POC authors fell into this category. In each 
case, either the POC author did not provide an e-mail address or the provided address was 
invalid. A postal letter or FAX message was used in these situations. To minimize the 
difficulties which arise in identifying missing or invalid e-mail addresses, POC authors 
could be required to select a desired method of contact (e-mail, FAX or letter) during the 
initial submission of their paper information. This would place the responsibility on POC 
authors to choose a method and to ensure a valid address or number was provided. 
Administrative personnel could then sort the database into the separate methods and 
proceed more easily with the acknowledgment process. The final submission application 


provided this option with “e-mail” set as the default method. 


b. Require POC Authors to Complete an On-line Mailing Label 


Issuing final acceptance notifications required formal letters to be mailed; 
however, for a large portion of the paper records, incomplete address information had been 
provided by the POC authors. This led to a significant delay in mailing the notifications 
since administrative personnel had to verify each address individually for use in a form 
letter. With the many overseas addresses, this became particularly troublesome when trying 
to merge the different address fields from a database record into a single address for a label. 
To avoid this problem, the on-line submission application could require authors to fill out 
an on-line mailing label, in a single text field, with the address entered exactly as it should 
appear for a letter addressed to the POC author. This requirement would readily support 
rapid form letter generation using a mail merge feature, which is built in to most modern 
word processors. This requirement was also added to and supported in the final submission 


application. 


OF PROPOSAL FOR A FULLY AUTOMATED SUBMISSION PROCESS 


This section contains a discussion covering the decisions and actions which would 
be necessary to support fully automated submission and acknowledgment of papers during 


both the initial and final submission phases. By completely automating these phases, a 
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conference planning committee could essentially eliminate the adminjstrative support which 
currently dominates the personnel and expense resource requirements for managing a 
conference. In order to completely eliminate administrative personnel requirements from 
these processes, the entire burden of responsibility for submitting and confirming receipt of 
papers would have to be shifted to the authors. By taking into account the current 
prototype’s requirements and making some hard decisions, the functions currently 
performed by administrative personnel could be either eliminated or converted into author 
requirements. Adding the following requirements to those already supported in the final 
submission application would provide full automation from the perspectives of the host 
institution and the conference planning committee. 


iG Require On-line Submissions 


Every paper would have to be submitted on-line using the submission application. 
No other form of submission would be allowed. This means that no FAX, no e-mail, no 
FTP and no postal mail submissions would be accepted. The “First Call For Papers” would 
make this a clear requirement and no postal address would be provided. This would require 
authors from around the world to have or to get access to the Internet via a browser in order 
to submit their papers. Administrative personnel would no longer be required to process 
papers sent by FTP, FAX, e-mail or postal mail since the information for all papers would 
be entered into the database and the papers would be loaded to the server by the authors 
over the Internet. 


2s Require PDF Submissions 


Every paper would have to be already converted into Adobe’s PDF format and 
would have to be readable using the English version of Adobe’s Acrobat Reader application 
before on-line submission. Since the PDF format includes compression and encoding 
appropriate for use over the Internet, no other compression or encoding scheme would be 
applied to the PDF files before transmission. Accepting only PDF submissions would 
require all authors to have or to get access to Adobe’s Acrobat Exchange software in order 


to convert their papers into PDF format prior to submission. Although this requirement 
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potentially imposes an added expense on authors, the gains on the administrative side would 
be enormous. Administrative personnel would no longer have to decode, decompress or 
convert received papers into the PDF format. Also, no scanners and no printer would be 
required. Most importantly, as will be discussed later, administrators would no longer be 
required to confirm the condition nor acknowledge the receipt of submitted papers. 


a: Require a POC E-mail Address 


Every POC author submitting a paper would have to provide a valid e-mail address. 
Acknowledgments and preliminary notifications would only by sent by e-mail to the POC 
authors. No postal or FAX messages would be provided. The “First Call For Papers” 
would also need to list this requirement along with other submission requirements for 
prospective authors. This would require all POC authors to have or to get an electronic mail 
account to participate in a conference. Authors would need to carefully enter their e-mail 
addresses to ensure they could be contacted when needed. The on-line paper record would 
have to allow author editing of the e-mail address field to support address changes. A test 
feature could be included which would allow authors to click a button to have the server 
automatically send them a test e-mail message within a certain period of time. As a fallback 
communications measure, acknowledgment and acceptance notification information could 
be automatically stored in the database and displayed from within on-line paper records. 
This would require POC authors to access their paper records on-line to receive the 
messages. | 


4. Automatically Acknowledge Each Submission 


Every single attempt at submitting a paper would require an automatically generated 
and transmitted e-mail message to the POC author. In addition to validating the POC 
author’s e-mail address, this acknowledgment message would include all identifying paper 
and author information. Consideration could be given to also including the password 
entered by the author so that the message would contain all essential information required 
for regaining access to the paper record in order to perform editing and/or future 


resubmissions. Also included would be instructions for accessing the actual PDF paper to 
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confirm its condition by simply opening and reading it on-line using, the English version of 
Acrobat Reader. In addition to eliminating manual acknowledgments, this requirement 
would also eliminate administrative confirmation of paper conditions because authors 
would be responsible for submitting and confirming receipt of their own papers. Allowing 
authors to view their papers exactly as they would appear to reviewers would serve to 
indicate that the papers were, in fact, successfully uploaded. Papers could be resubmitted if 
authors did not like the appearance or could not open their papers on-line. A major 
advantage in allowing authors to confirm their papers is that they are much better judges of 
how their work should look than administrative personnel who would only be able to 
confirm that the papers were readable, but could not validate the actual content. An added 
benefit would be that the number of resubmissions would probably decline as a result of the 
rapid feedback and the authors’ direct involvement in confirming their submissions. 


Oe Use the Paper Receiving Directory as the On-line Directory 


The same file directory used for receiving papers would have to be used as the on- 
line paper directory which provides Internet access to the submitted papers. Without this 
requirement, an administrator would have to move the received papers into the on-line 
directory on a regular basis in support of the automatic e-mail acknowledgments described 
above. With this requirement, authors would be able to access their papers after each 
submission. In fact, authors would be able to confirm their papers immediately after 
submission by providing them with on-line access instructions at the conclusion of their file 
transfer. This would provide even faster feedback and would further reduce the probability 
of additional and unnecessary resubmissions because authors would be able to complete 
their submission and confirmation requirements in a single on-line session. 


6. Use a Dedicated Server Exclusively for the Papers 


To properly support a fully automated paper submission process, a dedicated server 
would be required to receive and distribute submitted PDF papers. The file upload CGI (the 
Perl script) would have to run on this server in conjunction with a Web server to support 


receiving of the papers. The Web server would also provide on-line access to the received 
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papers. The requirement to separate the paper server from the database server would more 
efficiently distribute the increased load which would result from allowing authors to read 
their own papers on-line. All authors would upload their papers for submission and 
download their papers for confirmation which would essentially double their paper server 
access requirements. The high probability that most paper submissions would occur right 
before the deadline (as with the ISCAS conference) serves to highlight the necessity of this 
requirement. The server distribution would prevent the higher bandwidth requirements 
from the actual submission and confirmation of papers from interfering with the submission 
of paper and author information into the conference database. Furthermore, the server 
distribution would support the paper review process equally well since the paper server 
would provide reviewers with access to the papers while the database server would handle 
the receipt and processing of the on-line review forms. 


oF Automatically Disable Paper Submissions at the Deadline 


When the paper submission deadline is reached, new paper submissions would have 
to be automatically disallowed. ‘This would require displaying some form of a-“Deadline 
has Passed” notice when the submission application is accessed. Resubmissions would be 
allowed for an additional period of time to ensure that all last minute submissions could be 
confirmed and/or resubmitted if necessary. Both the submission deadline and the 
resubmission deadline could be controlled via preferences stored in the database which 
could be set or modified using a system preferences application (which would also be 
available on-line). Without this requirement, administrative interaction would be necessary 
to disable the submission and resubmission applications when the deadlines passed. 


8. Provide Extensive On-line Help 


To ensure that the submission and acknowledgment aspects of the initial and final 
submission phases remain fully automated with no administrative support would require an 
extensive use of on-line instructions and help files. The author requirements would have to 
be clearly described. Links to other resources, like Adobe’s Web site, would also be 


helpful. In addition, an automated password retrieval function would be essential. To 
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provide this capability, an application could be developed which would allow an author to 
submit a form containing a paper number to request a forgotten password. For security 
reasons, an e-mail message containing only the retrieved password would be automatically 
generated and transmitted to the e-mail address of the POC author on record for that paper. 
With more on-line help provided in the form of instructions, recommendations, technical 
information and technical support, administrative personnel requirements will decrease 
accordingly. 

Although the above proposal for a fully automated submission process is fine in an 
ideal sense, it may be unrealistic at this time to mandate that authors only submit papers in 
PDF format and that they have Internet access with a valid e-mail address. Conference 
organizers are interested in achieving high levels of participation and excessive 
requirements may threaten participation goals to some degree. The ISCAS conference, for 
example, achieved a world-wide level of participation. In fact, most authors were from 
foreign countries and, in some cases, did not have Internet access or e-mail accounts 
available for their use. In these cases, authors were only allowed to participate as a result of 
the postal submission option. 

Over time, advances 1n modern technology throughout the world should diminish 
the potentially negative impact of these automated on-line submission requirements and 
might, in fact, increase participation levels as a result of the increased accessibility. In the 
near term, however, a system which does enforce the automated requirements could be 
tested during a local or U.S. only conference in which it could be reasonably assumed that 
potential participants would easily meet the extra requirements. In the next chapter, 
conclusions for the research questions which provided the focus for this thesis will be 


addressed and recommended areas for future research will be described. 
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V. CONCLUSIONS 


The ISCAS ‘98 conference management prototype demonstrated that conference 
organization can. be effectively accomplished over the Internet. It further showed that an 
on-line collaboration system encourages participation of icmationally dispersed 
correspondents by providing an additional, near real-time means for communication. This 
final chapter will address the research questions originally targeted by the ISCAS prototype 
in order to summarize all conclusions which can be provided from the prototyping effort. 


Also included is a section on recommended areas for future research. 


A. RESEARCH QUESTIONS 


I, Which Conference Processes can be Web-enabled? 


a. Paper Submission and Acknowledgment 


The electronic submission of papers during both the initial and final 
submission phases is a process which is fundamental to any Web-enabled conference 
planning effort. Use of the Internet to support on-line paper submissions, which are linked 
directly to a database, provides extensive capabilities for tracking, sorting and managing the 
various data for later processes (like review, scheduling, registration, and publishing). 
Additionally, the use of a Web-enabled acknowledgment application allows administrators 
to prepare and transmit (via e-mail) paper submission confirmations with database 


information that is automatically merged into an on-line form letter. 


b. Paper Review and Selection 


Web-enabled processes can also support the efforts required to complete the 
review and selection of papers by the Technical Program Committee (TPC). Enhanced 
collaboration capabilities result from the integration of a Web server with a database of 
proposed conference papers. A Web application can interactively distribute the papers over 


the Internet to the reviewers and then collect their acceptance recommendations using a 
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Web form. Another Web application can then be used to summarize the information from 


the review forms for use by the TPC in making a final decision to accept or reject each 
paper. 
C. Presentation Scheduling 


The collaborative efforts required by widely dispersed TPC members to 
schedule accepted paper presentations into predetermined conference sessions can be 
significantly eased with an on-line scheduling application. With a database already 
containing all of the paper information (title, authors, etc.) and all of the pre-designated 
session information (dates, times, etc.), a Web application can be used to first select an 
appropriate session and then to assign papers by simply entering paper numbers. The key 
paper information can then be automatically displayed in the schedule from the database. 
This scheduling application can also support presentation order assignment and can prevent 
duplicate scheduling of papers. When scheduling is completed, the database can be used to 
format and print the final conference schedule. 


ap Which Conference Processes are Best Suited to Web Deployment? 


a. Paper Review and Selection 


Using the Internet as a communications medium for reviewing and selecting 
papers provides significant advantages when supported by a Web application with an 
integrated database. For the ISCAS prototype, this was the most successful process built 
into the on-line application suite. An enormous amount of time and money was saved by 
not having to copy and mail out the more than 1200 papers to the approximately 200 
reviewers who were located around the world. Additional savings were realized with the 
on-line review forms which were automatically collected and sorted into summaries for the 
TPC’s final selection process. In fact, it worked so well that TPC members did not have to 
meet in person until after the review and selection process was completed, and then only to 
identify and arrange the conference sessions. It is important to note, however, that Web- 


enabling this process would not have been possible without first getting all paper and author 
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information into a database and also providing electronic, on-line viewable versions of the 


papers. 
b. Paper Submission and Acknowledgment 


The processes for submitting and acknowledging papers are also well suited 
to Web deployed applications. | With the ISCAS prototype, the use of an on-line 
submission application was, in fact, critical for getting information into the underlying 
database which would be used during the review and selection processes, as well as all later 
processes. Also, the application used to acknowledge received papers allowed 
administrative personnel, working from any remote location, to send confirmations by e- 
mail, thereby resulting in significant savings in postage and handling requirements. 

There was a trade-off associated with using the on-line submission 
application. Supporting a postal submission option in addition to the on-line method 
provided backward compatibility; however, it also resulted in duplicate submissions from 
authors who used both methods. This, of course, placed an additional burden on 
administrators to eliminate duplicates from the database prior to using the acknowledgment 
application. This problem can be minimized by providing rapid feedback to the authors; 
however, it cannot be completely eliminated unless on-line submissions are the only option 
supported. Unfortunately, eliminating the postal option would limit participation to only 
those who have Internet access and e-mail accounts. 


ay: What are the Hardware and Software Support Requirements? 


a. Web Server 


A robust server platform capable of running standard HTTP compliant Web 
server software is essential; and, in fact, provides the foundation which supports all on-line 
conference collaboration processes. WindowsNT, MacOS and Unix systems are all viable 
server platforms which support a variety of hardware alternatives. There are many choices 
for Web server software as well. As mentioned in chapter two, Netscape Enterprise Web 


server (free for educational use) was run on an NT server in support of the ISCAS 
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prototype. Initially, an Apache Web server (free) was being used on a Linux (Unix) based 
486 PC. A paper collection and distribution server was later setup on an SGI from Silicon 
Graphics, which also used a Netscape Enterprise Web server. Finally, a MacOS system 
running Social Engineering’s Quid Pro Quo Web server (also free) was used during 
development and testing of the final submission application to avoid interfering with the 


production prototype. 
b. Database Server 


A data repository is critical to the conference planning effort and is 
necessary to provide a back-end to the Web server for supporting the on-line applications. 
Any ODBC compliant database server/application can be used. A Microsoft Access 
database was used for the ISCAS prototype on the NT server. The database server is only 
used as a data repository. All database management features (forms, joins, queries, 
indexing, etc.) are implemented using a middle-ware CGI. Development and testing of the 
final submission application was conducted using a demo version of the MacOS based 
Butler SQL database server from Everyware Development Corporation. The application 
was then loaded onto the NT server and linked to an Access database which used the same 


schema as the Butler SQL test database. 


c Common Gateway Interface 


Two different Common Gateway Interface (CGI) applications are required 
to enhance the functionality of a Web server in support of an on-line conference 
collaboration environment. First, a CGI is required to support file transfer (upload) from 
the authors’ TTSHTIRE to the conference Web server. For this, a Perl script CGI was used on 
the ISCAS server. The second CGI is needed to link the Access database to the Web server 
in order to support interactive data input and retrieval in addition to all other database 
management requirements. For the ISCAS effort, Tango for Access CGI (chapters two and 
three) was used to link with the Access database. Also, a demo version of Tango Enterprise 


CGI was used on a MacOS system for development and testing of the final submission 
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application. The Tango CGIs come with a companion application, Tango Editor, which can 
be used on either WindowsNT/95 or MacOS systems to build the on-line applications. All 
Tango applications have a common file format which allows them to be universally 


deployable on any Tango CGI platform (WindowsNT, Unix, MacOS). 
d. Other Support 


Several additional support resources are required to provide background 
support for a conference support system. An uninterruptible power supply (UPS) is needed 
to maintain system reliability and integrity. A tape backup system is needed to safeguard 
the contents of the Web site, the database, and the uploaded papers. Scanners are needed to 
convert mailed submissions into electronic PDF format for on-line viewing. A Postscript 
laser printer is needed to support various printing requirements. Also, Acrobat Exchange 
software (including Acrobat Distiller) is essential to support conversion of Postscript files 
into PDF format. 


4. Is Scalability an Issue With Regard to Hardware and Software? 


a. Platforms and Servers 


An on-line conference collaboration environment should be able to scale in 
accordance with the submission and review requirements necessary to support conferences 
of varying sizes. This is one of the advantages provided by a Web-based solution. The 
various conference functions could be supported by different platforms to more evenly 
distribute and share the load across multiple Web servers. With the ISCAS conference, for 
example, a 486 PC provided domain name support, limited static HTML Web page support, 
and mail services. A Windows NT server provided database services, on-line application 
services, and Web site services. And finally, an SGI provided on-line paper collection and 
distribution services. Any number of platforms running Web servers and CGlIs can be 
integrated into a linked Web farm to support a variety of conference collaboration services. 


For small conferences, all services can easily be provided from the same platform. 


1 


b. Middle-ware al 


The database middle-ware CGI should easily support the integration of 
additional conference support processes as needed. Many special purpose queries and/or 
capabilities will be required during different phases of a conference planning effort. Most 
of these will be related to retrieving and presenting information from the database. The 
Tango CGI provided a great deal of flexibility during the ISCAS effort because new 
applications could be developed and linked to the database independently of other pre- 
existing applications. This allowed new functions to be easily added to the conference 
application suite. Additionally, the Tango applications could link to any ODBC compliant 
database on any platform on the network, even to multiple databases from within the same 
application. This provides even greater flexibility for distributing services across multiple 
platforms and could play a significant role in providing secure on-line conference 
registration services. Unfortunately, time constraints prevented integration of a secure 


registration server for the ISCAS conference. 


c. Data Repository 


Adding new functions or processes in support of conference collaboration 
will require a database system with a flexible schema. A database must be able to support 
new data storage requirements as they are identified without impacting existing data and the 
applications which use them. For this reason, a relational database is well suited to a 
conference planning system, especially during prototyping. With a relational database, new 
tables can easily be added and linked to existing records using the indexes from other tables. 
This allows internal data structures to be scaled to meet requirements without affecting 
existing data. A flat-file database cannot be modified as easily since the addition of new 
data fields will require modifications to all existing records. Object oriented databases 
might provide additional flexibility; however, the technology is relatively new, so analysis 


in this area is left for future research. 
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=e Are There Any Special Considerations for a Web Deployed System? 


a. Stateless System 


A standard Web server maintains no information between client 
connections. Each Web page that is accessed by a client starts a new adnan with the 
server. Applications which run through a Web server, therefore, cannot store state 
information as they progress from page to page. This presents a problem when attempting 
to track a client’s progression through an on-line process; therefore, special care must be 
taken when developing Web-based applications to ensure that state information is 
maintained between connections. The stateless application problem can be overcome 
through the use of “browser cookies” which store state variable information in a client’s 
browser and through the use of hidden HTML input fields which can be embedded within 
on-line forms. Both of these techniques are employed throughout the ISCAS prototype to 


track user progression through the series of forms which compose the on-line applications. 
b. Client Configurations 


Developing applications to run over the Internet from a Web server requires 
consideration for all potential client browsers and their respective configurations. For the 
ISCAS prototype, this was particularly true with respect to proper functioning Web forms 
and accessibility to on-line PDF papers. Some older browsers would not properly display 
Web forms containing tags based on newer HTML standards; therefore, recommended 
browsers had to be listed on forms where advanced HTML features were essential. Also, 
browsers had to be properly configured with the appropriate plug-in software (Acrobat 
Reader) in order to access the on-line PDF papers. Several reviewers had to update their- 
browsers and configure them with the proper plug-ins to complete the on-line reviews. To 
minimize browser configuration concerns, Web pages and applications should be tested 


using as many different browser and client platform configurations as possible. 
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C. International Scope 


a 


When International participation is expected, an Internet based electronic 
collaboration system will result in some communications incompatibilities. With the 
ISCAS prototype, use of two byte fonts was a problem with many international paper 
submissions. Although the papers were written in English, the font systems used in many 
International word processors use two byte fonts that are incompatible with the single byte 
fonts in English versions of the software. Administrative personnel had to work closely 
with the authors of these papers to have the two byte fonts removed and to get the papers 


resubmitted in time for the review process. 


B. SHIFTING THE BURDEN OF RESPONSIBILITY 


The ultimate success of an on-line conference management system depends heavily 
on successfully shifting the burden of responsibility for administrative functions from 
conference administrative personnel to the authors who submit their papers for review. The 
authors have a vested interest in actively accepting additional responsibility during the 
initial and final submission phases. Since their objective should be to get their papers 
accepted and published, it is to their advantage to be proactive during these phases provided 
they are given appropriate guidance and access to the tools needed to support their efforts. 


At a minimum, the authors must be provided with the following capabilities: 


e The capability to enter their paper and author information directly into the 
conference database remotely via the Internet 


e The capability to electronically submit their papers via the Web 
e The capability to have protected access to their paper records 
e The capability to edit all information in their paper records to ensure its validity 


e The capability to resubmit their papers as many times as necessary to ensure a 
valid submission 1s available for review 
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@ The capability to acquire rapid feedback from their efforts 


e The capability to access their papers on-line to validate their submissions by 
viewing them exactly as the reviewers would during the review process 


By successfully shifting the burden of responsibility from a few administrators to, in 
this case, over 2000 authors, an on-line conference support system provides significant 
leverage to the conference planning and management effort. In fact, the more responsibility 
is shifted to submitting authors, the more automated the system can become from the 


conference planning perspective. 


. Be AREAS FOR FUTURE RESEARCH 
1. Scheduling, Registration and Publishing 


The ISCAS prototyping effort did not run to completion prior to publishing this 
thesis. As a result, the registration and publishing processes will be left for future research 
either with the ISCAS prototype or as part of another conference planning effort. As 
discussed in Chapter IV, integrating an on-line registration application for conference 
participants will provide several advantages during the publishing phase. Additionally, 
determining and implementing the specific requirements in support of an actual publishing 
effort would provide a more complete analysis ror an on-line conference collaboration 
system. Also, the manual process used to identify and schedule the conference sessions 
warrants close evaluation. Ideally, the process could be constrained in a manner which 
would allow it to be converted into an on-line process. This would eliminate the 
requirement to have the TPC hold a two day scheduling meeting, which was rather 
expensive considering that most TPC members had to be flown to Monterey. 


Ze Fully Automated Submission and Acknowledgment 


The final section in Chapter IV discusses some of the requirements necessary to 
support the development of applications which could provide full automation of the 


submission and acknowledgment functions during the initial and final submission phases. 
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Along with mandatory Internet and e-mail access, having authors submit their papers in 
English PDF format is one of the primary requirements for supporting full automation. 
Although it places the most difficult and/or expensive burden on prospective authors, 
mandating the use of Adobe’s PDF format for electronic paper submissions provides the 
most flexibility for an on-line conference support system and the greatest benefit to the 
conference planning committee. A great opportunity for a future research project would 
include using this requirement, in addition to the others described in chapter four, to 
develop a fully automated application suite for conference planning. 


oP ODBMS Conference Collaboration System 


The use of Object-oriented Database Management Systems (ODBMS) is on the rise. 
They represent the latest advances in database technologies. ODBMS systems have 
enhanced capabilities for storing non-standard data types, like multimedia objects. Building 
a conference collaboration system on top of an object-oriented database, as opposed to a 
relational database, would allow for storage of the PDF papers directly within the database. 
This could enhance the security and handling capabilities for storing and retrieving the 
papers. Further, this could provide significant advantages during the publishing phase, 
which currently will require papers that are stored in a directory to be correlated to their 
respective database entries before publishing. Also, the paper upload CGI, in addition to 
paper resubmission and file renaming requirements, would be completely eliminated since 
the database middle-ware would handle the task of storing the paper in the database where it 
would coexist with all related paper and author information. An ODBMS conference 
planning system would be an outstanding research project for anyone interested in learning 
about object-oriented databases and Web technology. 

4. Sealed Bidding and Other Government EC/EDI Processes 


_ The fundamental ideas and experiences gained from the ISCAS prototype can be 
abstracted into other applications of Internet/Intranet processes, especially those related to 
Electronic Commerce / Electronic Data Interchange (EC/EDI). The DoD’s sealed bidding 


process and other highly formalized Government processes are ideal for Web-based 
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implementation. In particular, sealed bidding provides an excellent opportunity for future 
research by an acquisition professional with an interest in Web development. The attached 
appendix contains a previously written document based on the ISCAS prototype which 
describes a DoD EC/EDI model for implementing an Internet browser-based sealed bidding 


process. It provides a solid foundation for beginning a future research project in this area. 
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APPENDIX. A MODEL FOR ELECTRONIC COMMERCE / ELECTRONIC 
DATA INTERCHANGE (EC/EDD IMPLEMENTATION WITHIN FEDERAL 
ACQUISITION SYSTEMS: AN INTERNET BROWSER-BASED SEALED 
BIDDING PROCESS 


James W. Coffman 
December 1997 
Naval Postgraduate School 
Monterey, CA 


Abstract: Most Federal EC/EDI initiatives rely on developing a new infrastructure to 
support the exchange of EC information. With any new communications architecture, 
systems integration and compatibility issues become a major concern. Additionally, 
undesirable restrictions are commonly placed on both Government and_ vendor 
organizations. The use of existing open data standards and the Internet architecture 
provides an alternative framework for EC/EDI implementation within Federal acquisition 
systems. A prototype was developed for the 1998 IEEE International Symposium on 
Circuits and Systems (ISCAS) to demonstrate and evaluate the use of Internet technology in 
providing an automated system for global information exchange and collaboration |Ref. 7, 
p. 1]. The ISCAS prototype is used as a model for describing a web-based sealed bidding 
process which addresses the objectives and intent of EC/EDI initiatives without incurring 
the problems and penalties associated with a new architectural solution. 


1. Introduction: The 1993 National Performance Review noted that Government must 
strengthen and broaden its EC/EDI capability, especially within Federal acquisition systems 
[Ref. 10, p. 39]. Electronic Commerce (EC) is the paper-less exchange of business 
information using electronic data interchange, electronic mail, electronic bulletin boards, 
electronic fund transfers and other electronic processes and technologies. Electronic Data 
Interchange (EDI) is the computer to computer transmission of a business document ih a 
standard format [Ref. 9, pp. 15-16]. It is generally accepted that EC/EDI provides a 
significant advantage to those who incorporate it into their business models. The Federal 
Government develops and maintains standard forms and documents which are typically 
used in paper format to conduct most or all Government related business transactions. 
These standard documents readily lend themselves to use in an electronic format within the 
EC/EDI context. 


2. Functional Requirements of the EC/EDI Process: In order to establish an EDI 
process, the information contained within a standard business document must be formatted 
into standardized, machine-processable data fields. These data field formats must be agreed 
upon and shared between all organizations which desire to participate in the EDI process. 
Furthermore, each organization must acquire or develop software applications which are 
capable of supporting the business field formats on the systems contained within their 
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Organizations. Once the data standards are in place and applications for processing the data 
are available, transactions can be conducted between the participating organizations by 
electronically transmitting standard business document formats [Ref. 11, p. 13]. 


The electronic transmission of standard business documents requires some form of a 
communications link. Existing EC/EDI initiatives routinely support one of the two 
common approaches for providing an electronic communications link in the commercial 
industry: the “direct connection” method or the use of a “Value Added Network (VAN)”. 
The direct connection method allows trading partners to transmit business data directly 
between their computers via commercial phone lines (modem to modem). The EDI VAN is 
a communications network that transmits, receives, and stores electronic business 
documents for EDI trading partners in a process which is similar to the use of an electronic 
mailbox. Using the VAN approach requires an additional communications architecture 
which alleviates most of the access restrictions inherent in the direct approach. The 
Department of Defense (DoD) has adopted a modified VAN concept for internal use which 
uses two Network Entry Points (NEPs) to allow all DoD activities to pass their EDI 
transactions through the Defense Information Systems Agency (DISA) to the commercial 
VANs that have registered with the DoD [Ref. 8, pp. 21-22]. 


3. Problems with Current Federal Initiatives: The use of an EC/EDI process within 
Federal acquisition systems should provide for full and free dissemination of acquisition 
information to the public, which includes all business concerns (large and small). 
Additionally, the EC/EDI process should not unintentionally exclude any public interest 
from participating in Federal acquisition business activities. With respect to technical 
feasibility, the entire public should have ready and easy access to the process without being 
encumbered with expensive, non-standard software and hardware systems acquisition and 
integration issues. By supporting these criteria, an EC/EDI acquisition process would help 
the Government maintain the competitive pricing advantages which are associated with a 
fully open and fairly competitive market. Presently, there is an architectural limitation of 
information, which creates an information barrier to entry, limiting competition [Ref. 9, p. 
116]. The following is a list of issues and concerns which are disadvantageous to the 
Federal acquisition process with regard to existing EC/EDI initiatives: 


a. The direct connection approach is only viable for activities doing business with just 
a few trading partners. An activity must schedule times for its partners to access the 
computer for transaction exchanges. As the number of partners and volume of 
transactions grows, the direct connection method becomes restrictive, cumbersome 
and expensive. For this reason, it is not a viable option for Federal acquisition 
activities [Ref.9, pp. 21-22]. 


b. The DoD VAN concept has created great concern with DISA’s inability to 
efficiently process the existing volume of DoD EDI transactions in a timely or 
consistent manner [Ref. 8, p. 23]. A significant amount of systems architecture 
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overhead has been added to the process. Additionally, the NEP concept provides a 
significant bottleneck which will deteriorate rapidly as more DoD activities begin 
using EDI. Any delays in processing EDI transactions will jeopardize competitive 
fairness, especially in regard to the issue of electronic bids and other time sensitive 
EDI transactions. 


c. The VAN method requires that all business entities desiring to conduct EDI 
transactions with a Government agency be registered with a VAN which is in turn 
registered with the Government agencies’ VAN [Ref. 8, p. 23]. The use of VAN 
services iS an added expense to the public business entity. This can have the 
unwanted effect of excluding small business concerns, especially those which would 
need to be registered with multiple VANs in order to conduct EDI business with 
multiple government agencies. 


d. There are numerous data standards in use for EDI transactions, with most activities 
using standards that are tailored to their transaction databases. This forces trading 
partner activities to use (share) the same data field standards. Since, as mentioned 
above, each activity must have applications which can work with the data, a 
significant cost 1s usually required for trading partners to develop special purpose 
EDI applications [Ref. 8, pp. 31-33]. This added expense can easily become cost 
prohibitive for small businesses, especially if more than one application is required 
to transact with more than one Government activity. There are additional life-cycle 
maintenance costs associated with these special purpose software packages. This 
has the added drawback of creating legacy (stove-pipe) systems which make it 
difficult for either the Government activity or the trading partners to upgrade their 
hardware/software configurations or to migrate to other platforms [Ref. 8, pp. 52- 
esi) : 


4. The Internet as an Alternative EC/EDI Communications Link: As much as the 
Government desires to uphold the sanctity of full and open competition in procurements, 
there should be a parallel full and open access to information. Such a conduit to 
information not only promotes full and open competition but also abides by the full intent 
and objectives of EC/EDI [Ref. 9, p. 116]. The Internet is an inexpensive (cheap) 
communications conduit which provides almost universal access. It is based on standard 
protocols which are widely accepted in the commercial sector. Most businesses. already 
have (or will soon have) Internet access. The browser software which is used to interact 
with the Internet is freely available to any user. 


Internet browser applications are based on a standard data formatting syntax known as 
HyperText Markup Language (HTML). Any individual or activity can create a web-site 
full of HTML documents (using any text editor) which can then be read by anyone using 
any standard browser application from anywhere on the Internet. A web-site requires 
another HTML-based standard software application, a web-server, which transmits the 
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HTML documents to all requesting clients (users with standard web browsers). A good 
web-server can easily support thousands of clients simultaneously. High quality browser 
and web-server applications are commercially (and often freely) available for all existing 
computer systems. 


As described above, a web-server typically only disseminates information to requesting 
clients. This is essentially a one-way communications link which does not fully meet the 
fundamental need for two-way EC/EDI communications. In fact, existing Federal Internet- 
based EDI initiatives tend to rely on web-servers solely for distributing information rather 
than also for collecting and processing data from clients [Ref. 9, pp. 118-120]. In order to 
collect data for a Federal acquisition process, the received client data requires security, 
integrity, time validity, client authentication and non-repudiation. In Implementing 
Electronic Data Interchange (EDI) at the Defense Fuel Supply Center, James M. Barnard 
discusses several techniques and methods which are now commonly used to resolve these 
client data requirements [Ref. 8, pp. 23-29]. In Addition to supporting these methods, an 
Internet-based bi-directional EC/EDI web-server must also be capable of retrieving the data 
from the clients. 


The HTML standard also provides syntax for creating electronic forms which accept input 
data from clients (via a browser application) for processing by a web-server middleware 
application. The typical web-server is only capable of receiving, not processing, the 
submitted form data. Two more software applications are required to complete the EDI 
communications link. A Common Gateway Interface (CGI) application is needed to 
process the received data; and a database application is required to store it. The web-server 
will pass received form data over to the CGI. The CGI will process it, store it in the 
database and generate an HTML response document which will be passed back through the 
web-server to the client. Database applications and fully configurable CGI (middleware) 
applications are also commercially available. Higher quality commercial CGIs are capable 
of interacting and working directly with most existing database systems used within Federal 
activities. It is important to note that all additional software requirements are placed solely 
on the server-side (Government) activity for an Internet-based EC/EDI solution. The 
standard HTML browser application is the only application required by the clients. Thus, a 
data-base enabled government web-site may provide a viable alternative to existing EC/EDI 
initiatives which does not incur the problems associated with the development of a new 
communications architecture. 


5. The Sealed Bidding Process: Developing a web-site to manage a local sealed bidding 
process would provide an ideal prototype for demonstrating the advantages of an Internet 
browser-based EC/EDI process. Such a prototype could easily be converted into a turn-key 
web system which is transportable and easily installed at any contracting activity on 
existing computer systems with Internet access. It is the formality of the sealed bidding 
process, as in many Government processes, which lends it to an EC/EDI implementation. 
Under the Competition in Contracting Act (CICA), procuring agencies are required to 
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procure “competitively.” The Federal Acquisition Regulation (FAR) closely governs the 
rigid requirements of the sealed bidding process. Its basic objective is to give all interested 
parties an opportunity to deal with the Government on an equal basis — with the 
Government theoretically reaping the benefits of full and open competition [Ref. 12, p. 3-2]. 
An Internet solution better supports this basic objective than do other EC/EDI 
communications frameworks. 


A sealed bid procurement begins with an Invitation for Bids (IFB), which is a collection of 
forms that includes instructions for bidders to follow, certifications to be signed, prices to be 
filled in, the technical requirements or specifications of the procurement, and both standard 
and special terms and conditions. The IFB is required to be distributed to a sufficient 
number of prospective bidders to ensure adequate competition. This is typically done with 
public displays, announcements and through the use of prospective bidders mailing lists. 
Any interested bidder can register for inclusion in these mailing lists. Mailing lists are 
currently the most effective means for publicizing a solicitation of bids. Although a long 
mailing list typically generates more competition, it also generates a significant mailing 
expense for the solicitation [Ref. 12, pp. 3-13,16]. Once potential bidders have received 
their bid packages, the forms (bids) are completed and returned prior to the stated deadline. 
Late bids are rejected unless they can be considered for award under the late bid rules. At 
the pre-designated time, the Government’s bid-opening officer opens and, when possible, 
reads aloud the most important terms of each bid. An abstract of all bids is prepared, which 
includes the bidders, their bids and other price-related factors which will be used to make 
the final bidder selection. All bids are then made available for public inspection. Award of 
contract is made to the responsible bidder who submits the lowest responsive bid. This 
entire sealed bidding process folds easily into an automated electronic bidding system 
which scales well with both the Government contracting agencies and the prospective 
bidding organizations. 


6. A Sealed Bidding Model for Internet EC/EDI: An electronic collaboration system 
has been developed which parallels many of the functions inherent in the sealed bidding 
process. This system is being used at the Naval Postgraduate School as a working 
prototype to coordinate and manage the IEEE’s 1998 International Symposium on Circuits 
and Systems (ISCAS ’98). The entire set of ISCAS conference processes is available on the 
Internet to provide a world-wide collaboration environment for the widely dispersed 
members of the committee and all conference participants. Although the ISCAS prototype 
does not currently support secure EC/EDI, it does provide a fundamental framework from 
which a comparative sealed bidding model can be abstracted (especially with the known 
ability to easily incorporate secure form processing into a web-site). 


The objective of the ISCAS °98 prototype was to pipeline, into a single on-line system, all 
of the traditional conference organization processes - particularly those related to receiving, 
processing, acknowledging, distributing, reviewing, accepting and scheduling papers 
submitted for conference presentations. In a typical conference, a first call for papers is 
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developed. The nature and objective of a first call for papers is not unlike that of an IFB (in 
relation to the respective functions of each). A first call for papers asks for paper 
submissions, discusses the submission requirements (including format), provides a 
preliminary list of topics, delineates a schedule (including deadlines) and describes the 
proper procedure for paper submissions. In a manner similar to bid solicitation, authors’ 
mailing lists (maintained by IEEE) are used for notifying potential authors and institutions 
of a pending conference by mailing out a first call for papers. Authors then submit their 
proposed papers for review by mailing them to the conference organizing committee at the 
host institution. The papers are received, verified, recorded, sorted and an acknowledgment 
of receipt is mailed to each author. As in the bid submission process, submitted papers are 
withheld from review until the deadline (bid opening) has passed. (At this point, the ISCAS 
and sealed bidding processes diverge in similarity, however, minor abstraction from 
existing on-line methods will allow visualization of a complete sealed bidding EC/EDI 
model.) Copies of the papers are then mailed out to the various reviewers for an acceptance 
recommendation. Completed review forms are mailed back to the organizing committee 
and a final acceptance determination 1s made. The accepted papers are then scheduled into 
related presentation sessions for the actual conference. 


A first call for papers (think IFB) was published on the ISCAS ’98 web site 
(http://www.iscas.nps.navy.mil) in HTML format for easy access via the Internet. Another 
electronic version of the first call for papers was also placed online in PDF format to allow 
for download and off-line viewing. Postal mail was used to distribute a paper version of the 
first call for papers, which directed potential authors (think prospective bidders) to the web- 
site. In future versions, technology will allow automated notification of a pending 
conference by automatically emailing notices to the entire authors’ mailing list (stored in an 
on-line database) at the time of a first call posting. The objective is to direct authors’ 
attention to the web-site. As authors become more familiar and comfortable with the 
process, they will begin to instinctively check the site for pending conference information 
(solicitations). 


On the web-site, approaching deadlines are highlighted in a rotating banner at the bottom of 
the screen. Hyper-links are provided to all of the requirements as well as to an online form 
which allows for Internet browser-based paper submissions (think sealed bids). This form 
is actually the first in a series of forms which capture the required identification and contact 
information along with author preferences for presentation method and paper topic / sub- 
topic categories. A time-stamp is automatically entered into the database along with the 
form data to record submission time. A unique tracking number is auto-assigned to the 
paper and returned to the author for future reference via an HTML results document. The 
final form allows an author to interactively select a paper to be submitted from a local hard 
disk. A “Submit” button is then pressed which transfers the paper (using the HTTP 
protocol) to the web-server. The web-server passes the incoming file over to a CGI which 
saves the file to a designated partition (protected) on the server. Although not fully 
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exploited in the ISCAS prototype, current technology allows auto-conversion of the 
incoming file into an Adobe Acrobat PDF file for online viewing (if it is not already in 
Acrobat format). A final HTML results document is then sent to the author to confirm 
receipt and to redisplay the paper tracking number for the author’s future reference to the 
paper submission. 


The paper submission now requires validation to ensure that it has been received in good 
condition (non-corrupted) and that it has been converted into a PDF file. Validation is 
conducted by administrative personnel who are not members of the ISCAS committee or 
review personnel. Online readability of the electronic paper is the only concern (not 
content). (/t is important to note that this step would not be required for a sealed bidding 
process. All sealed bid information could easily be obtained via machine-readable 
standard form fields rather than from a PDF document. If a PDF file were required, a 
method for on-line validation by the submitting bidder is an available option. This would 
allow a bidder to verify his/her own submission by opening the bid on-line immediately 
after transmission to prove that it was received intact. Security features are available to 
support this technique.) A successful validation results in an auto-generated email 
acknowledgment to the author, which identifies the paper by title, category, author(s) and 
paper tracking number. An invalid submission results in a negative email acknowledgment 
which warns the author (bidder) to resubmit using the same tracking number. Valid 
submissions are then transferred into a protected on-line directory to await the submission 
deadline (bid opening) which signals the start of the review process. When the deadline for 
submissions is reached, the paper submission application is taken off-line. This eliminates 
problems associated with late submissions (/ate bids) by simply disallowing them. The 
capability to completely automate this deadline action is available, although it was not used 
to terminate the ISCAS ’98 submission application. 


The paper review process begins by allowing protected, log-in access (via assigned 
passwords) for the paper reviewers. The reviewers are then emailed a list of assigned 
papers to be reviewed. An on-line review form is available which has a direct link to the 
paper being reviewed. The papers can be reviewed over the Internet using the Adobe 
Acrobat browser plug-in (free). 


As stated before, the review portion (as well as the remainder) of the ISCAS process is 
different from the sealed bidding process; however, the parallel to be drawn is that the - 
papers are made available on-line immediately after the deadline.(although access is 
restricted). With regard to sealed bidding, the bids themselves could be made available for 
public scrutiny (automatically) at the time set for bid opening. Furthermore, the abstract of 
bids could be auto-generated, sorted and auto-emailed to the designated “bid-opening 
officer” who could then access the on-line bids for a more detailed review (if necessary for 
contract award). 
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To complete the sealed bidding EC/EDI model drawn from the ISCAS comparison above, 
an application could be developed to allow on-line award of contract. This application 
would allow designation and confirmation of the winning bidder by selection of the bid 
tracking number which identifies the responsible bidder who submitted the lowest 
responsive bid. The notification process would also be completely automated (via email) 
for notifying the winner and other bid participants of the award. In addition, an on-line 
announcement could be auto-generated for public access. 


7. Summary: The sealed bidding model described above effectively demonstrates that the 
existing Internet infrastructure, when combined with open standards (HTML), provides a 
credible alternative to most current Federal EC/EDI implementations. Unlike current 
initiatives, which tend to restrict competition, an Internet browser-based EC/EDI process 
may actually increase competition. The sealed bidding model shows that there are many 
advantages to an Internet-based solution, including (but not limited to) the following which 
are particular to sealed bidding, but could easily be extrapolated into other Government 
EC/EDI processes: 


1. Anentire Internet-based sealed bidding EC/EDI application suite could be developed in 
a few weeks using currently available commercial software. 

2. The sealed bidding applications could be installed on any Unix, Windows NT or 
Macintosh based web-server and could then be connected to any existing database 
system with the requisite data fields. 

3. The database system can reside on any platform on any part of the Local Area Network 
or anywhere on the Internet. | 

4. Prospective bidders could register for inclusion in electronic mailing lists with the use 
of an automatic online registration application, which would be included with the sealed 
bidding application suite. 

5. A significantly larger pool of prospective bidders would be able to compete at no added 
expense to either the Government agency or the bidders. 

6. No more expensive postal mailings would be required of the contracting agency in order 
to distribute IFBs. 

7. A sealed bidding web-site would allow for near real-time distribution of IFBs and bid 
submissions. 

8. Bidders could receive instant feedback as to the receipt status of their bid. The HTML 
results document returned from the CGI could securely display enough information to 
confirm to the bidder that the bid had been posted in time. In fact, a signed and 
irrefutable certificate could be returned from the web-server to the bidder. The bidder 
would need to save the certificate for legal purposes and proof. 

9. Bid openings could be automated with instantaneous online publishing of bid 
information at the exact time specified for bid opening. 

10. An abstract of bids could be automatically generated at bid opening time and could be 
sorted according to price and price related factors. 


90 


Ie 


Ie 


Is: 


14. 


i> 


The online database could automatically flag known non-responsible bidders in the 
abstract of bids. 

Security and authentication / non-repudiation technologies can be integrated into 
existing web-servers and web-browsers. Electronic encryption and identification 
certificates can be registered and plugged directly into the software. 

The sealed bidding submission application could be configured to automatically start 
and stop receiving bids at designated times (i.e., automatically stops allowing 
submissions at bid opening time). This would help in eliminating the late bid problem. 
Sealed bidding system modifications would only be required on one platform (the web- 
server). This means that changes would be immediately available to all clients 
(prospective bidders) who access the web-site after the changes. (Migration and 
scalability would no longer be a major concern.) 

Changes in requirements / IFBs can be automatically sent (in near real-time via email) 
to all bidders who have already submitted bids by simply using the database of existing 
bids (bidders) to filter the database mailing list. 


The ISCAS °98 prototype is a proven system which provides a solid foundation for 
formulating the sealed bidding EC/EDI comparison and for drawing the above conclusions. 
There are currently more than 1200 paper submissions on the web-server. There were 
approximately 2300 contributing authors from more than 60 countries. The various paper 
reviewing committees totaled more than 200 members from around the world. The review 
process was completed with more than 2400 electronic reviews submitted. Very little 
money has been spent on postage by the conference organizing committee in support of the 
paper submission and review processes. 
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